report resukes 

EO 017 985 24 ea ooi it6 

TRAIMINO EFFECTS OF FEEDBACK AND MODELINO PROCEDURES ON 
TEACHING PERFORMANCE. 

BY- MCDONALD, FREDERICK J. ALLEN, DWIOHT W. 

STANFORD UNIV., CALIF., SCHOOL OF EDUCATION 

REPORT NUMBER BR-5-1030 PUB DATE 67 

CONTRACT OEC-6-10-078 

EDRS PRICE MF-$1.00 HC-$9.36 232P. 

DESCRIPTORS- ^TEACHER EDUCATION, ^TEACHING SKILLS, ^FEEDBACK, 
TRAINING LABORATORIES, TEACHER BEHAVIOR, VIDEO TAPE 
RECORDINGS, ^TEACHER INTERNS, LITERATURE REVIEWS, SELF 
EVALUATION, SUPERVISORS, ^REINFORCEMENT , MODELS, STATISTICAL 
ANALYSIS, LEARNING PROCESSES, QUESTIONNAIRES, EFFECTIVE 
TEACHING, STANFORD, 

THIS REPORT DESCRIBES A SERIES OF EXPERIMENTS TO ASSESS 
THE USEFULNESS OF TELEVISION RECORDINGS IN IMPROVING TEACHING 
PERFORMANCE. OBJECTIVES OF THE STUDY ARE (1) TO COMPARE THE 
EFFECTS OF SELF-EVALUATION OF A TEACHING PERFORMANCE WITH 
FEEDBACK PROVIDED BY A SUPERVISING INSTRUCTOR, <2) TO COMPARE 
THE EFFECTS OF REINFORCEMENT DELAY, AND (3) TO COMPARE THE 
EFFECTS OF A PERCEPTUAL MODELING DEMONSTRATION OF A DESIRED 
BEHAVIOR WITH THOSE PRODUCED BY PROVIDING A WRITTEN 
DESCRIPTION OF THE BEHAVIOR AND TO COMPARE THE EFFECTS OF 
COMBINING REINFORCEMENT WITH EACH. EACH OBJECTIVE WAS THE 
SUBJECT OF A SEPARATE EXPERIMENT CONDUCTED UNDER HIGHLY 
CONTROLLED, LABORATORY-LiKE CONDITIONS. IT IS CONCLUDED THAT 
THE RESULTS OF THIS STUDY SUPPORT THE ASSUMPTION THAT THE 
RATE AND LEVEL OF LEARNING A GIVEN TEACHING STRATEGY VARY AS 
A FUNCTION OF THE MODE OF MODEL PRESENTATION. THERE IS 
EVIDENCE TO INDICATE THAT PERCEPTUAL MODELING PROCEDURES ARE 
CHARACTERIZED BY DISTINCTIVE CUING PROPERTIES WHICH TEND TO 
RECOMMEND THEM OVER SYMBOLIC MODELING PROCEDURES FOR USE IN 
TRAINING CONTEXTS ANALAGOUS TO THOSE DESCRIBED IN THE 
EXPERIMENT. (HW) 
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INTRODUCTION 



The set of experiments reported here represents an attempt to 
analyze the effects of reinforcement and modeling variables on the 
learning of a selected class of behaviors, called "technical skills 
of teaching." These experiments were conducted at Stanford University 
using the trainees in the Secondary Education Teacher Program. 

We think these experiments are unique in that they are one of 
the few, if not the only, attempt to modify teaching behavior by 
using the paradigms of psychological experimentation. Each of these 
experiments might be seen, then, as a way of modifying teaching be- 
havior as well as a test of the comparative effects of different kinds 
of procedures. Obviously, this comparison is the essence of the exper 
imental work presented here. 

The experiments also introduced certain innovations in experim- 
ental technique as applied to research on teaching behavior. Rather 
than use large scale experiments under regular classroom conditions, 
many of the experiments were conducted under highly-controlled , lab- 
oratory-like conditions. In two of the experiments, observations 
were made of teaching behavior in classrooms, but the actual treat- 
ments were conducted under highly-controlled conditions. We call 
attention to these aspects of the experiments since other investigator 
may want to adapt our procedures. To the degree that we have demon- 
strated their feasibility, others will be less reluctant to depart so 



radically from the usual practices in investigating teaching behavior. 

We would also like to call attention to the innovative use of 
the videotape technology. All of this experimentation was made poss- 
ible through the use of this device. Whether or not similar kinds of 
experiments could be conducted without it is a moot point. However, 
these experiments illustrate that the videotape recorder may wall be 
the technological instrument which will substantially improve exper- 
imentation on teaching behavior. 

These experiments also make a contribution to the development o£ 
psychological theory. The variables used in the experiments repre- 
sent variables which are currently being investigated by a la r^ number 
of researchers, though the behavior being studied is considerably more 
complex. To the degree that it is, these experiments represent gener- 
alizations of what has already been learned, and as will be seen when 
the results are read, suggest modifications of some concepts. 

We would like to thank the Stanford Interns who served as the sub- 
jects in these experiments. Although the experimentation provided 
them with training, it nevertheless placed great demands on them. 
Without their assistance and cooperation the experiments would not 
have been possible. 

We would also like to thank particularly Dr. Michael E. J. Orme, 
now of Harvard University, who as a graduate student served as a prin- 
cipal research assistant on this project. Many individuals, working 
as assistants, organized our data by rating videotapes and by super- 
vising experimental sessions. Each of these persons, by his effective 
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work made possible the successful conduct of the experiments, and 
deserves our gratitude even if it can be expressed in so modest a 
form as this acknowledgment. 

We would also like to thank Dr. Robert N. Bush, who encouraged 
our experimentation, and who made possible its conduct in the Teacher 
Education Program. His continuous support of our ideas and our work 

was a necessary condition for its success. 

Dean I. J. Quillen's support and his provision of time and re- 
sources to conduct these experiments were of inestimable value to us. 

Frederick J. McDonald 



Dwight W. Allen 
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CHAPTER I 



PROBLEM AND RATIONALE 



This report describes a series of experiments to assess the use of 
television recordings to improve teaching performance. The availability 
of portable videotape recording devices has made it possible to study 
teaching behavior in new ways. Contemplated uses could not be made pre- 
viously either because recording equipment limited what could be done or 
costs were prohibitive. 

Problem 

One of the most difficult problems in designing instructional 
systems to produce teaching behavior has been to provide adequate 
feedback information on the teaching performance. The student or intern 
teacher judges for himself his own performance on a day-to-day basis. 

He is periodically, though not frequently, "supervised". 

Usually, the character of the supervision is left to the discretion 
of the supervisor, even though supervisor training procedures may have 
been casual and sporadic. In the last analysis, it is the supervisor 
who observes, records, and reports to the teacher. To utilize- this 
feedback the teacher has to see himself as somebody else has seen him, 
and he must relate these perceptions and judgments to his own. The 
difficulties in this procedure are that it invites heavy reliance on pri- 
vate frames of reference, the communication requires a high order of 
psychological skill, and it stimulates defensiveness. 

All present methods of giving a teacher information about his per- 
formance have one or more of the following problems; (1) they inade- 
quately control the defensive reactions of communicator and communicatee; 
(2) they require a teacher to visualize his performance from a word de- 
scription of how he is behaving; (3) they do not begin from a common 
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perception of what was donej why and how it was done, and what the 
effects were. 

The responses of students in his class provide a teacher with 
another kind of feedback. This feedback, however, may be quite diverse, 
some students responding positively or negatively for reasons unrelated 
to the quality of the teaching performance. Students also mask their 
reactions. Test results, another form of student feedback, are explain- 
able only in part by the effectiveness of the teaching. Although student 
reaction is important despite these inadequacies, the beginning teacher 
may be insensitive to it or may grossly misinterpret it. Even if the 
beginner is learning to interpret student reaction, he has difficulty 
relating it to the specific behavior that produced it. It seems clear 
that both student reaction and supervisor’s comments lack the important 
characteristics that make a feedback process effective and efficient in 
producing the learning of teaching behavior. 

The problem is to provide adequate feedback to control and to 
facilitate the acquisition of effective teaching behavior. The feed- 
back process should have the following features: (1) the feedback pro- 

cess should reproduce the teaching performance as completely and as 
reliably as possible; (2) it should be as objective as possible; (3) it 
should be as immediate and as frequent as possible. 

An audio-visual recording of teaching behavior has the first two 
characteristics. The third characteristic is acquired by the scheduling 
the viewing of the performance as soon after it occurs as possible and 
by having the recording available for frequent viewing. 

Audio-visual recordings of a performance have the following 
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additional advantages: (1) the teacher can observe his own performance; 

(2) when a supervisor talks to the teacher about his performance they have 
a common starting point. The first of these advantages presumably re- 
solves the problem of the teacher visualizing his performance. The 
second advantage should tend to minimize, or at least put some reality 
bounds", on the defensive reactions of the teacher and supervisor and 
provide for the establishment of a common frame of reference. 

Another problem in the learning of teaching behavior is adequately 
representing the desired behavior. One way teachers learn about desired 
performances is by observing model teachers . These observations are 
usually uncontrolled; that is, the beginner may observe whatever he chooses 
to observe, or may not see what he has been told to observe. He may not 
interpret what he observes correctly. The method of observation does not 
provide for successive examinations of the teaching performance, nor for 
careful analysis of it. The same difficulties plus the effects of for- 
getting and selective remembering characterize the trainee's recollections 
of teaching performances he has observed during his own schooling. Ob*' 
viously, descriptions of how to teach are fraught with all the problems 
of verbal description. 

Audio-visual records of teaching perfbrmancfes ihay be controlled in 
many ways through editing. The desired display is literally created. 
Positive and negative instances of the behavior can be provided. The 
effects on student behavior can be correlated with the teaching behavior. 

A variety of instances demonstrated by many teachers under different 
circumstances can be shown. The display can be repeated and can be used 
in a critical analysis of the performance by instructors. 
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Audio-visual records, specifically videtape records, provide 
greater control over two processes known to be related to effective 
learning, demonstration of the desired behavior and feedback on a per- 
formance in which the behavior is attempted. However, basic research is 
needed to assess the relative importance of the two processes, their 
relation to the efficiency of the learning, and the influence of vari- 
ables associated with each process affecting its effectiveness. The 
experiments described in this project are designed to assess the rela- 
tive effects of various arrangements of these two processes. 

Although television recordings seem to offer many advantages there 
is little information available on the use proposed here. We do not 
know how effective watching one’s own performance may be, to what degree 
the observation needs to be structured, whether teachers' reactions to 
the observation will facilitate or hinder learning, and what kind of a 
demonstration of the desired behavior, if any, needs to be included. In 
short, the characteristics of an efficient demonstration and feedback 
process need to be determined. 

Similarly we do not know how to talk to teachers about their per- 
formance when a visual record is available as the material of the discus- 
sion. Will there still be difficulty in establishing a common frame of 
reference? Will the teacher and supervisor even agree on what they see? 

Is pointing out adequate performance sufficient? 

Other problems are that we do not know the contigency schedule that 
is most effective with this kind of feedback. Nor do we know what behavior 
patterns are most likely to be affected by it. 
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A consent on the technology of the portable television tape re- 
corder is relevant. The portability of the recording apparatus en- 
courages its frequent and regular use. Elaborate arrangements do not 
have to be made to record a class session. At Stanford we have developed 
a self-contained portable cart which can be installed in a classroom 
during the passing period between classes. Interference with classroom 
routine is minimized. The recordina process is relatively inexpensive. 

The major advantage from the viewpoint of providing feedback is that 
the recording is immediately available. A teacher could, for example, 
though this is not at present being recommended, teach a part of a class 
hour, step outside and watch his performance, and return to his class 
for another performance hopefully improved from his observations. He 
may also study his own or another's performance under supportive and 
instructive conditions. 

Related Research 

The paucity of research on how teachers learn to teach is best 
illustrated by the limited attention devoted to this topic in the 
Handbook of Research on Teaching (Gage, 1963). Historically, the major 
research method has been to correlate measures of teacher characteristics 
with ratings on effectiveness. Stern, summarizing the literature, notes 
"no substantial progress along these lines, despite the marked trend 
toward the use of more reliable measures of predictor variables." (Stern, 
1963, p. 419). One of the major difficulties encountered in these studies 

has been the ambiguity of the criterion. 

Another line of investigation has related specific teacher behaviors 

to student behavior. Flanders, studing classroom climate, has developed 
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THREE MAJOR CATEGORIES OF TRAINING VARIABLES 




Along each axis are four subcategories of variables. Thus, feedback 
may be either positive or negative, and may be either self- or other- 
administered. The proposed experimentation explores variations along 
each axis and the interaction among the major categories. 



a recording procedure for analyzing clasaroom interaction (Flanders, 1960 ). 
In this method, observers classify teacher and student talk. Flanders 
found that achievement was higher in classes in which teachers used in- 
direct influence, and that students praised by the teacher received more 
sociometric choices. (Flanders, 1960 ) The purpose of the Flanders’ 
studies was to assess interaction effects. They are part of a series of 
such studies (Anderson, 1945 ; Withall, 1949 , 1956 ; Thelen, 1959 ). These 
studies, however, are not training studies. They, like those discussed 
in the first paragraph, are designed to show the relation between classes 
of teacher behavior and classes of student behavior. They are consistent 
with the notion that if certain kinds of teacher behaviors can be learned, 

certain effects will be produced. 

Figure 1 presents a taxonomy of the variables which are relevant 

to the problem being studied. The major categories of variables are 
those associated with the feedback, practice, and demonstration or 
guidance systems which are the major components in training systems for 

learning many teaching performances. 

Each of these in turn has been subdivided into the major categories 

being studied or proposed for study. For example, practice of a teaching 
performance may be conducted in either a controlled situation such as 
microteaching, where the teacher instructs a small group of students 
for a short period of time during which he attempts to enact one set of 
teaching responses, or he may practice the teaching skills in a classroom 

under the conditions prevailing there. 

Feedback may be positive or negative, where positive feedback has 

informational characteristics, i.e., appropriate responses are indicated 
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as appropriate j such reinforcement may also be rewarding That s very 
good." Negative reinforcement consists in indicating inappropriate re- 
sponses or omissions; such reinforcement may be mildly adversive -- 
"That's not a very effective way of teaching that." 

Either kind of feedback may be mediated by another person, or the 
feedback may be mediated through a self-critique of one’s teaching per- 
formance (in our experiments the use of the videotape recorder has made 
feasible the detailed criticism of one’s own performance after the teaching 
event has occurred; the record is also a complete one so that the super- 
visor or teacher does not have to depend on his memory of the teaching 
performance . 

The demonstration variable may also be broken down into several basic 
ways of portraying a desired response. The category, symbolic demonst^- 
tion , consists in descriptions of the desired behavior, either written or 
spoken. In such demonstrations the subject does not view an actual por- 
trayal of the desired behavior. The label, perceptual demonstration (in 
these studies it has also been called percept ual modeling^consists in por- 
traying the desired behavior for the learner. Such portrayals may be live 
or mediated by videotape. In the experimental studies conducted in this 
project the modeling of desired teaching behavior has been mediated by 
videotapes. Each of these forms of demonstrating may be subdivided into 
two kinds. For example, a subject may view the best instances of his own 
teaching performance. This would be an instance of self -modeling or a 
self-perceptual demonstration. Similarly, a subject may write out for him- 
self a detailed plan for enacting a performance which he thinks is the best 
possible approach to handling a problem. (We have not yet used this form. 
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but its use seems to be feasible.) Others may give instructions (symbolic 
demonstration), or may portray the behavior (perceptual demonstration). 

A review of the relevant literature by Bandura and Walters has shown 
that complex behavior may be acquired almost entirely through imitation 
(1963b). They state that the provision of face-to-face models serves to 
accelerate the learning process and in cases where errors are dangerous 
or costly, becomes an essential means of transmitting behavior patterns 
(Bandura and Walters, 1963b, p. 52). In addition, Bandura, Ross and Ross 
(1963c) have demonstrated that film-mediated models are as effective as 
real-life models in transmitting some patterns of behavior. 

These experimental demonstrations of modeling effects have used 
young children as subjects and aggressive behavior as the dependent 
variable. While it has been generally speculated that similar effects 
occur with adults and that these effects cover a wide range of dependent 
variables, research literature in this respect is meager. One problem 
for future research is to find the modeling variables which increase the 
learning efficiency. It has been shown in other research that it is 
possible, using modeling procedures, to produce learning in which the 
subjects have acquired the behavior to the same level as the models 

(McBrearty, Marston, Kaufer, 1961). 

A brief theoretical discussion of the determinants of observational 
learning seems appropriate. Miller and Dollard (1941) assume that such 
learning is contingent upon the administration of reinforcing stimuli 
either to the model (M) or the observer (S) . Mowrer's (1960) proprio- 
ceptive feedback model also highlights the role of reinforcement in 
imitation learning. He distinguishes two types of such learning. In 
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the first, S is reinforced directly. In the second, S receives vicarious 
reinforcement. This empathic learning, as it is termed, is assumed to 
occur when M exhibits responses which are reinforced and S in turn em- 
pathically experiences reinforcement as well. Hill (1960) uses this 
conception in generating a model of the identification process. 

Bandura and Walters (1963b), however, point out that these imitation- 
reinforcement theories do not account for the learning of matching re- 
sponses when S does not perform M responses during acquisition, or when 
reinforcement is not provided for either M or S. The acquisition of 
imitative responses under these conditions can best be accounted for in 
terms of contiguity theory as developed by Sheffield (1961). It is 
assumed that when S observes M exhibit a sequence of responses, S acquires 
through the contiguous association of sensory events, perceptual and 
symbolic responses possessing cue properties that are capable of eliciting 
at some time after demonstration, overt responses corresponding to those 
that have been modeled. On the basis of this kind of reasoning, Bandura 
(1963b, 1965) has suggested that the acquisition of model behaviors 
occurs through contiguity, while reinforcements administered to M exert 
their influence on the performance of imitatively learned responses. 

His findings support the notion that under conditions where children are 
exposed to aggressive models who are either reinforced, punished or left 
without consequences, reinforcement acts as a performance-related variable 
rather than a learning or acquisition one. Children in the model-punished 
conditions did not imitate the model nearly as often as those in the other 
conditions. When an attractive incentive was later offered to induce all 
three groups to reproduce the model’s aggressive behavior, the previously 



- 10 - 



observed performance differences were completely wiped out. 

It cannot be assumed from the above discussion that the mere exposure 
of S to M constitutes a sufficient condition for imitative learning in a 
complex training sequence. Bandura (1964) states that factors other than 
contiguity undoubtedly influence imitative response acquisition. 

One of the major factors associated with learning is reinforcing 
learning behavior (a discussion of this literature appears later in this 
section). Of interest is the problem of the relative effectiveness of 
reinforcement and modeling. The relevant research on this point is 
presented first. 

Bandura and McDonald (1963a) have shown that under certain conditions, 
providing models is significantly more effective in changing behavior 
than are procedures in which only reinforcement is used. They designed an 
experiment to test the relative efficacy of social reinforcement and 
modeling in modifying moral judgment responses. One group of children 
observed adult models who expressed moral judgments counter to the group's 
orientation. These children were socially reinforced for adopting the 
models' evaluation responses. A second group observed the models but 
received no reinforcement. The third group did not view the models, 
but were reinforced for moral judgments that ran counter to their domi~ 
nant evaluative tendencies. Following the treatment, subjects were 
tested for generalization effects. The treatments, counter to Piaget's 
age-specific hypothesis, produced substantial shifts in the children's 
moral judgment responses. As the authors predicted, modeling cues 
proved more effective than operant procedures. The provision of models 
alone was as effective as the combination of modeling and reinforcement. 
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As Bandura and McDonald point out, ths failure of operant procedures 
to produce change is not surprising considering that the desired responses 
were very weak to begin with. In many cases the desired responses oc- 
curred so infrequently that there was little opportunity to influence them 
through reinforcement. Thus, even though the behaviors to be learned in 
a training experiment were already in the subject's repertoire, one might 
expect that reinforcement alone would be considerably less efficient than 
a procedure which was designed to highlight salient cues that preceded 
the desired behavior. 

The problem of the studies described here is to assess the relative 
influence of demonstration and feedback variables on learning specified 
classes of teaching behaviors. The effects of one of these variables, 
feedback, have been extensively investigated under such labels as rein- 
forcement, knowledge of results, effects of praise and blame, and trial 
and error learning. From Thorndike's formulation of the Law of Effect 
to Skinner's analysis of operant conditioning and despite theoretical 
differences, a large body of psychological literature supports the 
generalization that reinforcement procedures produce learning. 

Knowledge of results experiments differi.in the way in which the 
knowledge of results are obtained. In a target shooting experiment, 
for example, the subject obtains information directly related to cor- 
recting his performance. In experiments involving verbal learning, the 
reinforcement procedure not only confirms correct responses but also 
provides an opportunity for correction of errors and an implicit-practice 
trial (Michael and Maccoby, 1953, 1961). 
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Although these components of feedback have not always been parceled 
out, the results of the experimentations have been consistent; increased 
knowledge of results facilitates learning. However, in a complex learning 
task such as learning a teaching behavior, the relative effectiveness 
feedback may be highly dependent on the kind of feedback provided. A 
teacher, for example, who watches his own performance is getting feed- 
back, but its effectiveness may be limited compared to a confirmation 
procedure in which the response is confirmed and its behavioral history 
is discussed. Similarly, the effectiveness may vary between a supervisor 
confirmation and an observation of the immediate effect on student behavior 
A related issue concerns the immediacy of the feedback. Porter has 
pointed out that in human learning experiments a "true" comparison between 
immediate and delayed reinforcement has yet to be made (Porter, 1957). 
Because experimental control is not readily available in a classroom, the 
' question of amount of delay of reinforcement becomes paramount. Video- 

tape makes possible comparisons of relatively short delays to delays of 
hours and days. If appropriate conditions can be arranged, a teacher may 
step out of a class after a short teaching session and view his perfor- 
mance; or, he may view it at the end of the hour, or before his next 
teaching period, or even several days or weeks later. Each of these 
variations modifies the procedure so that more may be learned from the 
feedback. The usual inhibitory effects of delay may be diminished under 
> these conditions. It is also possible that delay may be desirable as 

a means of increasing the objectivity of the teacher who may be less 
j defensive in his analysis of performance after a lapse of time. If this 

is true the videotape procedures can provide a unique combination of 
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faithful reproduction, common frame of reference, and objectivity of the 
feedback. 

Exposing a person to a complex sequence of stimulation is no guaran- 
tee that he will attend to the entire range of cues, will select from the 
total stimulus complex the most relevant stimuli, or that he will even 
perceive accurately the cues to which his attention is directed. Sheffield 
and Maccoby (1960) have demonstrated that increasing the distinctiveness 
of relevant modeling stimuli greatly facilitates observational learning. 
Their work suggests that a key problem in observational learning is 
stimulus -discrimination learning. 

According to Wulff and Kraeling (1961), one way to clarify the nature 
of the stimulus event in associative learning is to propose that a learner 
makes implicit responses to the individual features of stimulus objects 
(which function as stimulus events), and that those implicit responses 
become associated with overt responses . The stimulus features which elicit 
such implicit responses are termed cues . 

For training research, the important implication seems to be that the 
treatment applied should not only require the trainee to focus on the cor- 
rect end-response, but also require him to respond to salient cues that 
occur during the course of learning. Lumsdaine (1961) reflects this 
concern in suggesting that programmed learning specialists have been pre- 
occupied with reward schedules to the detriment of the manipulation of 
prompting cues. Thus, undue attention has been paid to corrective feed- 
back procedures, and the guidance of learning has been neglected. 

It is obvious that the playback features of the TV unit can be used 
to develop procedures that emphasize cueing, since a tape can be stopped 
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or replayed at any point in the playback. Thus Doth prompting and dif- 
ferential reinforcement techniques in discrimination training may be in- 
corporated in training sessions. There are also implications here for 
testing the differential effectiveness of symbolic and perceptual modeling 
procedures. Before they are drawn, a word might be added concerning 
another aspect of the relationship between prompting and the reinforcement 
or confirmation elements in a training sequence. 

Findings obtained by Cook and Kendler (1956) and subsequently con- 
firmed by Cook (1958) for paired-associate learning indicate that when a 
prompting procedure is pitted against a no-prompting confirmation pro- 
cedure, the former is superior. Angell and Lumsdaine (1960) have obtained 
evidence that a mixture of prompted and unprompted trials is more effec- 
tive than exclusive use of prompted trials. More recently, these 
authors have demonstrated that when adequate prompting is provided, varia- 
tions in the kind of confirmation or feedback given are much less impor- 
tant than when less prompting is provided (Angell and Lumsdaine, 1961). 

What are the implications of this research for studies of the effects 
of symbolic and perceptual modeling? In general, perceptual modeling 
should be superior to sjnnbolic modeling in a videotape training session. 

It is reasonable to assume that in a perceptual modeling condition, the 
experimenter would have more opportunities to provide discrimination 
training on relevant cues since the model could be trained to emit the 
criterion behavior frequently. Further, in the perceptual modeling con- 
dition there is relatively greater emphasis on prompting techniques. 

Having viewed the prepared tape beforehand, E could say to the sub- 
ject, "Look, the pupil is about to respond (say, by asking a question). 
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The model is going to give both non-verbal and verbal reinforcement for 
this." In the symbolic modeling procedure, greater emphasis is placed 
on confirmation techniques since E and the trainee view the latter's 
playback, not a prepared model tape. 

Finally, Sheffield's (1961) discussion of perceptual blue-printing 
would lead one to predict that a relatively greater emphasis on viewing 
a model over viewing the self would produce the greatest increase in 
response strength for a given class of behaviors. This perceptual blue- 
print notion is like Tolman's (1959) conception of a cognitive map in 
that both serve to organize discrete stimulus-response elements in a 
given task. The advantage of such an organized sequence is that it 
provides a distinctive stimulus context within which appropriate response 
items can be specifically cued (Sheffield, 1961, p. 29). 

This means that in a training situation where the responses required 
are already part of the subject's repertoire, the problem becomes one 
of increasing the saliency of the perceptual aspects of the treatment 
The perceptual modeling condition appears to be superior to the symbolic 
modeling condition in this regard because the model teacher can be trained 
until clear-cut and unambiguous displays of the criterion behavior are 
filmed. This display includes the verbal and non-verbal cues that sig- 
nalled the occurrence of the criterion behavior. In this context, per- 
ceptual adequacy is of prime importance. 

Other things being equal, perceptual modeling should be a more 
efficient training procedure than symbolic modeling for the following 
reasons: perceptual modeling procedures permit one to display a large 
number of the desired responses; they increase the distinctiveness of 
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relevant stimuli; and in training sessions E uses them to highlight 
cues and responses by prompting techniques. Lastly, it is suggested 
that the perceptual adequacy of such modeling facilitates the development 
of perceptual blue-prints which serve to unify discrete elements of the 
desired skill. 

The major issue is whether feedback needs to be as extensive if 
relevant response demonstration is available. The theoretical issues 
and related research have recently been analyzed by Bandura and Walters 
(1963b). In a series of experiments, Bandura has shown that the availa- 
bility of models portraying a behavior class significantly influences 
behavior change. Bandura and McDonald have further shown that exper- 
imental conditions where models were present significantly changed behavior 
over the condition where only reinforcement was used in behavior shaping 
(Bandura and McDonald, 1963) . 

One reason advanced for the superiority of the modeling conditions 
is that it eliminates the relatively lengthy period of trial and error 
and shaping of successive approximations. Another reason is that the 
model *s behavior heightens awareness of the relevant cues to which 
responses are to be attached, again eliminating the trial and error period. 
There is also little ambiguity about either the character of the response 
or its effectiveness. 

In demonstrating teaching behavior, it would be expected that the 
effects of modeling would be heightened. The cues are more diverse and 
more subtle. The desired response is not easily evoked partly because of 
its complexity, partly because the learner has never initiated the response 
under a similar set of conditions. The model is known to be portraying 
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the desired behavior, a condition not explicitly present in other 
experiments . 

The experimental literature suggests that both modeling, a form of 
demonstration, and feedback will be effective. But as noted earlier, 
their relative effectiveness needs to be assessed. The human learner 
because of his greater information-processing capacities may require less 
feedback when the desired response is modeled. The feedback may be more 
effective when delayed because greater use can be made of this information- 
processing capacity. However, the responses to be learned may be of suf- 
ficient difficulty to require some combination of demonstration and feed- 
back, to need both a portrayal of the desired response and a period of 
corrected practice. 

The studies described here are experiments to assess these relative 
effects and to study some of the variables associated with both processes. 
Both their theoretical and practical significance is considerable. A 
feedback procedure can be expensive in time and materials. If the infor- 
mation-processing capacities of the human learner can be used to shorten 
this procedure, a finding in this respect would be highly useful in 
developing an instructional system for teacher training. Determining 
the most effective forms of feedback would be similarly useful. Such 
results would also contribute to the analysis of the specific contribution 
of feedback procedures in human learning. 

The Hunter College Study of the Improvement of Student Teaching 
utilized television recordings to improve the quality of instruction 
given by student teachers. (Schueler, ^ al.- » 1962) The television 

record did not significantly improve student teaching when used as a 
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part of the supervisory process. The authors of the study discuss several 
reasons why results probably were not attained, among them the range of 
variation in the teaching situations. Although this project has been 
successful in developing a meaningful and reliable measuring instrument, 
the procedure sampled a relatively large range of teaching behaviors. 

The extensiveness of the changes required may not have been achievable 
in the period of student teaching, at least not in sufficient amount 
to yield significant differences. Similarly, for a large sample of be- 
haviors, carefully described, the visual record may not add enough to pro- 
duce significant changes. Although verbal descriptions are not usually 
satisfactory, the instrument used in this study describes behavior spe- 
cifically, and the behavior frequencies are tabulated for the inspection 
of the student teacher. The problem seems to be to determine what the 
visual record adds to the learning conditions. 

Objectives 

1. To compare the effects of self-evaluation of a teaching performance 
with feedback provided by a supervising instructor. (Experiment I, see 
below) Does a learner need the cueing properties provided by an observer 
uhen his own performance is displayed to him? 

2. To compare the effects of delay of reinforcement and the kind of 
reinforcement provided (Experiment II, see below) The effects of delay 
may be reduced when the feedback is more comprehensive. 

3. To compare the effects of a perceptual modeling demonstration of a 
desired behavior with those produced by providing a written description 
of the behavior (symbolic modeling), and to compare the effects of com- 
bining reinforcement with each. (Experiment III, see below) The 
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desired behavior may be learned more easily from written descriptions; 
however, most theorists believe that observing the model is necessary to 
learn the cues to which' the desired behavior is attached. 

The experiments described attempt to answer three questions which 
reflect these objectives: (1) What is the relation between a modeling 

demonstration and a feedback procedure to the acquisition of a teaching 
skill? (2) What is the effect of varying the source of feedback, self 
or other, provided? (3) What is the effect of varying the amount and 
kind of feedback? 

Procedure 

A series of experimental studies are described here, each in detail 
below. These studies have several characteristics in common. Each 
utilizes a before-after design. The base rate of the behavior to be 
learned is measured in pre-experimental videotape recordings. This 
measurement is necessary since the desired behavior is likely to be already 
occurring in some form and at some frequency level . 

Each experiment limits the behavior class to be learned. A specific 
behavior, for example, reinforcing student participatory responses, is 
the behavior to be acquired under one set of experimental conditions. 

This limitation effectively eliminates the problem of attempting to 
change a wide range of teaching behaviors and of measuring these changes 
under the rubric of general teaching effectiveness. 

Experiment 1; A Comparison of Self-Evaluation and Social Reinforcement 
on the Acquisition of a Teaching Behavior. 

a) Teaching interns were randomly assigned to one of four experimental 
conditions. The independent variable was Self -Other evaluation of 
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the teaching performance. The dependent variable was the frequency 
of the desired behavior, approval of participating response by 
students . 

In the first condition, the teaching interns viewed their own per- 
formance, counting the number of approving and disapproving responses. 

In the second condition, a supervising instructor viewed the intern's 
recording with him, and reinforced each desired response by general 
approval, as it occurred. In the third condition, the supervisor provided 
general approval but also pointed out the effects of the teacher s 
approval on student participation and suggested ways of improving it. 

This arrangement was feasible because of the playback feature of the 
videotape mechanism. In the fourth condition, the control group, sub 
jects viewed their own performances but were given several categories 
of teaching behavior to analyze. 

The rationale for these treatments was that the supervisor-evaluation 
conditions cued the subject to the character of the desired response, and 
in the third condition pointed out the consequences of the teacher behavior 
These conditions provided specific and controlled feedback. A competing 
hypothesis was that if the subject was directed to observe and count his 
own behavior he would provide his own feedback. The experiment provided 

a test of these two competing hypotheses. 

Three evaluation sessions were run for each subject, and the time 
interval between viewing and the next recording was held constant. Sub- 
jects in the control group were also recorded three times with the same 

time intervals intervening. 

b) The sample of subjects were first year graduate students enrolled 
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In a teaching intern program at Stanford University. All were beginning 
teachers . 

c) Data gathered; A pre-experlmental, post-experimental, and two 
rexperimental recordings were made of the teaching behavior. All re 
cordings were made in the Interns ' classrooms . Each of these records 
was analyzed by pairs of independent observers who did not know the treat' 
ment given each subject. These observers counted the number of the 
desired responses. This number was then treated as the score for that 

session. 

d) Analysis: A mixed model, analysis of variance design was used 

to analyze the data, with rows standing for individuals, and columns 
for treatments, and each cell containing the replications. The .05 
level of significance was chosen for rejecting the null hypothesis. 

Experiment II: Effects of Feedback and Practice Conditions on the 

Acquisition of a Teaching Strategy 

The treatment of supervision sessions was alike for all subjects 

in that they viewed videotaped playbacks of their earlier teaching per- 
formance with an experimenter who provided discrimination training. How 
ever, instead of varying the amoung of feedback within each sessron as 
in the first experiment, we held withln-sesslon feedback constant, and 
manipulated the amount of practice and delayed feedback over four experl 
mental groups. In addition, the post-test was videotaped approximately 
seven weeks after training so as to permit Inferences about retention 
as well as acquisition curves. 
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The teaching behavior to be learned in this experiment was a 
redirecting and refocusing technique for handling students' questions. 

The purpose of the technique was to encourage the student to answer a 
question for himself or to correct his response, in contrast to giving 
an answer or correcting one. 

Three experimental sessions were run for each subject with time 
intervals between viewing and recording sessions controlled. Subjects 
were randomly assigned to conditions: 

b) Sample of subjects: In all of these experiments the same sample 

of subjects were used. 

c) Data gathered: Again, pre-experimental , post-experimental, and 

two experimental sessions were recorded. Independent observers counted 
the number and kind of question-handling techniques. The number of each 
kind was used as a score for each individual. 

d) Analysis of data: As in Experiment I. 

Experiment III: The Effects of Modeling and Feedback Variables on the 

Acquisition of a Complex Teaching Strategy. 

One of the objectives of the study was to compare two modes of 
model presentation in training sessions for intern teachers. The exper- 
imental design tested the hypothesis that the rate and level of learning 
a given teaching skill varies as a function of the mode of model presen- 
tation. Two types of modeling were considered: (1) Symbolic Modeling: 

This is defined as a process whereby one transmits desired behaviors to 
the learner by means of written or verbal instructions. The subject does 
not view an actual portrayal of the desired behavior. 

(2) Perceptual Modeling : This is defined as a process whereby one 
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transmits desired behaviors to the learner by means of a filmed model 

who portrays the desired behavior. 

Three experimental sessions were run for each subject. Subjects 

W8r8 assign8d randomly to troatmonts. 

a) A new sample of subjects, the second class were used in this 

experiment. These subjects had the same general characteristics as the 

subjects described in the first two experiments. 

b) Data gathered: The same general procedures described in the 

preceding experiments were used in this experiment. The behavior class 
of interest is the probing behavior described in Experiment II 

c) Analysis of Data; As in Experiments I and II 
General Considerations, 

1) In each of these experiments a single teaching behavior was the 
object of study. No brief Is made that these were the most Important 
skills to be learned, nor that their acquisition would guarantee teach- 
ing success. They were chosen because they represented generalized skills 
that a teacher probably needs to have In his repertoire. Experience 
with beginning teachers has Indicated that they have difficulty In 
performing these behaviors. They typically give more disapproving 
responses than approving responses to student participatory behavior, are 
more likely to answer a question than prompt a student to answer It, 
or direct *e question to another student. They tend to confine their 
questioning and prompting to a limited number of students. 

What has been learned from these experiments about the acquisition 
of these skills may be generalized to similar skills. The variables 
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that affect the learning of the simple form of the skill presumably 
would influence the learning of the more complex variation of the skill. 

2) A portable videotape recorder produced by Machtronics, Inc , 

Palo Alto, California was used. This instrument has been used for two 
years in pilot studies and for clinical training. The equipment can 
be set up in a classroom between periods, and operated quietly. Its 
small size renders it unobtrusive. 

3) Both teachers and classes were run through a familiarization 
period before the actual filming of experimental recordings. Experience 
has indicated that students largely ignore the filming when the equip- 
ment has been present for several days. The interns became accustomed 
to using the recordings during the summers preceding the experi- 
ments. During this period they became accustomed to seeing themselves 
on videotape. 

4) Length of viewing samples was standardized for all experiments. 
Conditions for viewing were also standardized. Experience with the 
recorder has familiarized the camera crew with desirable recording angles, 
which. were also standardized^ 

5) Reliability of scoring was computed from the ratings of the 
independent observers. A training period was provided with films al- 
ready available. Experience with these ratings yielded high observer 
agreement . 

6) Specific directions were given to experimental subjects on the 
general characteristics of the lesson to be recorded. These directions 
precluded recording study sessions or hour lectures with no student 
participation. 
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In the following chapters, each of the experiments is described 
in detail. The design and results of each experiment are presented 
separately. Tables presenting the data accompany the description of 
the results. Materials used in the experiment, such as directions given 
trainees, may be found in the back of this report together with addi- 
tional statistical information. 



CHAPTER II 



EXPERIMENT I 

THE EFFECTS OF SELF -FEEDBACK AND BUEINFORCEMENT ON 
THE ACQUISITION OF A TEACHING SKILL 



Materials for this study may be found in Appendices A-E. 

I 
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The experiment described here applies well-kncwn principles of 
reinforcement theory to a training problem. The training paradigm in- 
volves applying a reinforcer to an emitted response. The prediction 
is that the rate of responding will increase. 

Two aspects of the training procedures do, however, depart from 
those typically used in reinforcement studies. First, the reinforcer is 
not given while the learner is actually emitting the response. In this 
study, the subjects were videotaped while emitting complex responses 
(teaching) some of which were to be reinforced. After the actual 
behavior sample was collected the subject viewed his performance in 
the presence of an experimenter. When the desired responses appeared 
on the videotape, the experimenter reinforced their occurrence If 
results similar to those obtained in other studies occur in this si- 
tuation, reinforcement concepts are widely applicable. Also training 
procedures or conplex skills can be developed which use these principles 
and concepts in ways directly analogous to the procedures used in labo- 
ratory studies where the utility of these concepts has been amply 
demonstrated. 

The second characteristic of the training procedure which departs 
from the usual laboratory methods is that the behaviors to be learned 
occur in the context of many other behaviors and are relatively more 
complex than operants conditioned in laboratory studies. They are more 
analogous to the kinds of behaviors that have been verbally conditioned 
in psychotherapy sessions. Here, as in these other complex verbal in- 
teraction, it is literally impossible to reinforce every instance of 
the operant being conditioned. The immediacy of the reinforcement, for 
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similar reasons, is also somewhat variable. Again, if comparable results 
are obtained, the generalisability of reinforcement concepts is supported. 

Since the subjects in this experiment are humans it was also possible 
to test the efficacy of a cue-discrimination procedure This procedure 
consisted of pointing out to the subjects those cues to which the rein- 
forced operant was attached. In this way the cue-response chain is 

clearly indicated which should be facilitating. 

Since human subjects are being used, it is also possible that they 
can reinforce themselves or, more generally, provide their own feedback. 
Videotape recordings of a trainee's behavior sequence can be given 
to him so that he can view himself as a behaving organism. With instruc- 
tions he can note the presence or absence of the desired response. How- 
ever, it is dubious if such a procedure is likely to be highly facili- 
tating since the subject may not attend well, may be easily distracted, 
may be highly subjective in his viewing. The effectiveness of this train- 
ing procedure remains to be tested, however. 

This experiment, then, tests the relative effectiveness of three 
training procedures, each representing an application of reinforcement 
principles. The three procedures represent points on a continuum from 
self-administered feedback or reinforcement to experimenter -administered 
reinforcement with cue-discrimination training 

The prediction is that the order of training effectiveness will be 
in the same direction, with the self-administered feedback the least 
effective and the experimenter-administered feedback with cue- discrimin- 
ation training the most effective. 
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METHOD 



General Procedurt ; Intern teachers were videotaped on four separate 
occasions during the first 20 minutes of regular classroom lessons. In 
the intervals between each of these taping sessions they received differ- 
ential feedback as part of their regular supervision. The treatment or 
supervision sessions were alike for all subjects in that they viewed 
videotape playbacks of their earlier teaching performance. The mode 
and amount of feedback given each intern was varied by manipulating the 
reinforcement and discrimination training provided by an experimenter. 

Before the pretest videotapes were recorded, all subjects were told 
when they would be taped and were asked to present a discussion- type 
lesson in which teacher-pupil interaction could be observed. This was not 
a new or unusual experience for them as they had been frequently exposed 
to the videotaping-playback-supervision process during the previous three 

months of the Stanford Intern Program. 

Pupils in each of the classrooms were informed by the interns before 

hand that the portable TV equipment would be present in the room, and that 
the cameras would be focused on the teacher, not the class. 

Treatments : Mode of feedback, type of reinforcement and amount of 

discrimination training were varied for four experimental groups of 
interns . 

Controls: Group (C) : At the beginning of the first playback session 

these subjects were given written instructions which suggested that as 
they viewed subsequent playbacks of themselves, they try to determine 
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their effectiveness in relation to; the aims of the lesson; use of 
examples; effectiveness of teacher-questions; amount of pupil participation; 
pacing of the lesson; and teacher-pupil rapport. Following this, they 
viewed the first and all subsequent playbacks alone. E started the 
machine and left the room. He returned as the tape finished, stopped the 
machine and told the intern when to expect the next taping and the date 
for the following playback session. As for all subjects. Group C 
(controls) viewed playbacks of the preceding lesson within three days of 
its taping. Lessons were videotaped within two days of each playback 
session. 

Self “Feedback; Group (S-F) : These subjects followed the same basic 

schedule as the controls, except that they received a different set of 
instructions. The instructions discussed the educational rele- 
vance of increasing pupil participation in certain types of lessons; 
defined pupil participatory responses (PPR) as a clearly observable non- 
verbal or verbal response that was considered desirable; and provided brief 
examples of such behavior. It was also suggested that the intern immediately 
reward a PPR when it occurred as this would tend to increase pupil par- 
ticipation. Examples of teacher responses - both verbal and non-verbal- 
were then provided. Finally, a simple rating chart was attached so that 
the intern could classify his responses to PPR-s as "teacher rewards"; 
"teacher ignores"; "teacher punishes"; or "can't classify." As with the 
control group, self -feedback subjects viewed each playback alone. 
Reinforcement -Only Subjects; Group (R) ; These interns received the same 
written instructions as those in the self-feedback condtion. However, 
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E viewed each of the three videotaped playbacks with them, and verbally 
reinforced all observable instances of these Group (R) interns rein- 
forcing PPR's. Whenever the intern was observed to reinforce a PPR, E 
responded by saying "Good'."; "That's it!"; etc. Beyond this, he did not 
comment upon the intern's teaching performance. 

Reinforcement Plus Discrimination Training; Group (R + D) : These interns 

were first given written instructions which were identical to those ad- 
ministered to Group (S-F) and Group (R) subjects. In viewing subsequent 
playbacks with them, E provided differential reinforcement as in the 
Re inf or cement -Only treatment. In addition, he provided discrimination 
training. This consisted of pointing out salient cues to which reinforce- 
ment should be attached, suggestions related to the immediacy, affect- 
loading and types of reinforcement the teacher could use, and finally, the 
effects of such behavior upon pupil participation. In general terms, then, 
it might be said that Group (R + D) subjects received "maximum supervision" 
and Group (S-F) subjects received "self -supervision" . 

Experimental playback sessions for all groups were thirty minutes 
in length. When E viewed playback with Group (R) and (R + D) subjects, 
he reduced the sound momentarily when providing discrimination training, 
and spoke over the tape when providing reinforcement. Interns in all 
groups were informed that they could have the tape stopped or reversed and 
played over again at any point during the playback. Playbacks were stopped 
occasionally by E when he was working with the supervised groups. However, 
since the sessions were limited to thirty minutes, only one or two brief 
stoppages in sessions two and three were possible. 
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Subjects ; 

All Stanford intern teachers preparing to teach English, social 
studies and mathematics were included in the study. Approximately equal 
numbers cf interns from each of the subject-matter areas were assigned to 
each group. In addition, groups of interns teaching in the same school 
were distributed throughout the four groups. In this way, systematic 
bias due to subject-matter major or pupil characteristics based on socio- 
economic status was avoided. The major characteristics of the sample 
are summarized in Table 1. (See Table 1, Experiment I). 

The Dependent Variable ; 

The dependent variable was defined as the relative frequency with 
which the teacher positively reinforces pupils' participatory responses 
during teacher-pupil interaction in the classroom. PPR's were defined 
in training sessions with the interns as any desirable or releva n t pupil 
comment, answer or question. For purposes of measurement however, desir- 
ability and relevancy were not considered. 

The basic strategy in defining the dependent variable involved 
classifying teacher responses into one of four major response categories. 
These include positive reinforcement, negative reinforcement, interaction 
and information-giving responses. Pupil responses were also classified 
and considered in relation to teacher responses. Each of the above 
response categories are defined in the following discussion. A summary 
of the classification system appears in Table 2. (See Table 2, Experiment 

I)^ 
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TABLE 1 (EXPERIMENT I) 



MEAN AGE AND FREQUENCY DISTRIBUTIONS FOR SEX, 
GRADE-LEVEL TAUGHT, AND SUBJECT-MAJOR FOR 
EACH EXPERIMENTAL GROUP 



N = 71 



Expex’imental 

Group 


Mean 


Sex 


Grade 


Level 


Sub.1ect-Ma.1or 


Age 


Male 


Female 


9-10 


11-12 


English 


Soc.St. 


Math 


Group 1 
Controls 
(N = 18) 


2H.9 


h 


iH 


13 


5 


9 




5 


Group 2 
Self-Feedback 
(N = 18) 


2H.6 


5 


13 


13 


5 


7 


7 


k 


Group 3 
Reinforcement 
Only 
(N = 18) 


2H.H 


5 


13 


11 


7 


5 


9 


h 


Group H 

Reinforcement + 
Discrimination 
Training 
(N = 17) 


2H.0 


3 


iH 


12 


5 


8 


8 


1 
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Teacher Positive Reinforcement; a teacher response was defined as pos- 



itively reinforcing if it met one of two conditions: First, the response 

had to immediately follow a PPR - Secondly, it had to be classifiable under 
one of the following response categories. (1) Teacher Positive Verb al 
Rp Enforcement (+VR) : Immediately following a PPR, the teacher uses words 

and phrases such as "Good," "Finel" (2) Teacher Positive Non-Ver^ 
Reinforcement (+NVR) . The teacher in responding to a PPR, nods, smiles, 
leans or moves toward the pupil, or writes the pupil's response on the 
blackboard. (3) Teacher Positvel v Qualified (+QR) and (4) Post ,_ Ho c 
Reinforcement (PHR) . The teacher emphasizes positive aspects of pupil 
responses by reorienting class attention to earlier contributions by a 
given pupil (PHR), or by differentially reinforcing the acceptable com- 
ponents of a partially adequate response (+QR) . 

Teacher Negative Reinforcement : A teacher response was defined as nega- 

tively reinforcing if it immediately followed a PPR, and was classifiable 
as the obverse of one of four types of reinforcement outlined above (-VR; 

-NVR; -QR) . 

Teacher-Pupil Responses Independent o f Reinforcement Classification s: 
Certain responses that occur frequently in interaction and yet are not 
classifiable as some form of reinforcement were included in the definition 
of the dependent variable. These included inf orna tion-giving by the 
teacher, teacher-initiated interaction (i.e., questions directed to a 
given pupil or to the class in general), pupil initiated interaction (i e , 
volunteered comments or questions), teacher-no-response and pupil-*no- 



response . 



In general, the dependent variable included evaluative and informa- 
tional signals which the teacher may use in the classroom. There was a 
tendancy to emphasize socially rewarding operants since it could be 
expected that such behaviors would tend to increase pupil participation. 

Both in the training and measurement phases of the study, the PPR 
was presented as an which served to cue the teacher or rater that a 
desirable (or classifiable) teacher response was about to occur. 

Measurement Procedures ; Four videotapes for each intern in each group were 
analyzed by raters trained for this purpose. Throughout the rating phase 
of the experiment, they worked on the tapes in a random order so that they 
neither knew the treatment condition nor the number of the teaching trial 
of the tape being rated. Operators ran the television equipment and 
selected tapes using a list of random numbers. 

In addition to recording the frequencies of each of the behaviors 
defined as components of the dependent variable, the raters recorded 
other relevant behaviors and lesson characteristics as well. Frequencies 
were recorded for the total number of pupils who responded, the number 
of responses they emitted, and the sex of each responder. The raters also 
recorded the length of each videotape to the nearest tenth of a minute, 
and determined how much time was spent in discussion, group work or 
individual study. 

The Unit of Measurement ; In analyzing pupil-teacher interaction, one may 
record discrete responses, or measure in terms of some unit such as the 
uninterrupted utterance. Raters were trained to define an interruption as 
a comment or question. "Partial" responses that teacheis commonly emit 
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during pupil speech ("um-hum"; "yes," etc.) were not defined as interrup- 
tions. All forms of verbal and non-verbal reinforcement were scored in 
terms of discrete responses. For example, if the teacher said, "Good."j 
"Good!"; "That's fine'", the rater coded all three operants. 

Training of Raters ; Eight raters were initially given intensive training 
on intern videotapes . Once they had achieved at least 90/4 interrater 
agreement on all of the major response categories, and better than 95 ^ 
agreement on teacher reinforcement responses, the analysis of experimental 
tapes was begun. Reliability was maintained throughout the analysis by 
scheduling frequent joint rating sessions where raters checked the per- 
centage of agreement and referred to definitions of relevant responses so 
that systematic rating biases would not develop. Neither ratings taken 
during training, nor those produced in the joint sessions were used in the 
statistical analysis of results. 

The ratings upon which the reliability coefficients reported in Table 
3 are based were acquired in the following way. As each block of 30 or 
40 tapes were completed (a total of 269 were actually rated) , each of the 
six raters who did the bulk of the rating then rated a given tape. This 
was done without the rater's knowledge. Eight tapes, two from each trial, 
and two from each group were rated by all six raters in this way. As can 
be seen, interrater agreement is high. (See Table 3). 

A certain amount of data was lost between initial videotaping and 
the final statistical analysis. Some tapes were technically poor, and 
could not be rated. Some tapes were inevitably less than the required 
20 minutes, and were also omitted. However, if a given tape was over 
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TABLE 3 (EXPERIMENT I) 



INTERRATER RELIABILITY FOR SIX RATERS 
ON THE MAJOR RESPONSE CATEGORIES 
OF THE DEPENDENT VARIABLE 



Response Category 


Kendall Coefficient cf 
Concordance: W 


Positive Verbal Reinforcement 


1.00 


Positive Non-Verbal Reinforcement 


0.99 


Negative Verbal Reinforcement 


1.00 


Negative Non-Verbal Reinforcement 


0.87 


Total Pupil Reinforcement 


0.97 



15 but less than 20 minutes, the obtained ratings were prorated to a 
20 minute base. Of the original set of 284 videotaped lessons, 25 
were omitted at the outset, and 51 tapes were prorated before sta- 
tistical analyses were performed. The T statistic was used to determine 
whether or not certain cells in the matrix were biased by the inclusion 
of a disproportionate number of adjusted tapes. T was non-significant. 
Omitted and pi Drated tapes are shown by group and trial in Table 4. 

As can be seen, they are scattered throughout groups and trials. 

(See Table 4, Experiment I.) 
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TABLE 4 (EXPERIMENT I) 



NUMBER OF VIDLOTAPKU OMITTED 
AND PRORATED BEFORE STATISTICAL ANALYSIS 





Trial 1 


Trial 2 


Trial 3 


Trial 


. k 


TOTALS 


1 


Group 


P 


0 


P 


0 


P 


0 


P 


0 


Prorate 


Omit 


1 (c) 


2 


2 


2 


2 


1 


0 


2 


1 


7 


5 


2 (S-F) 


3 


2 


h 


1 


3 


0 


1 


2 


11 


5 


3 (R) 


6 


1 


1 


1 


5 


1 


6 


h 


18 


7 


It (E + D) 


h 


h 


2 


2 


h 


1 


5 


1 


15 


8 


TOTALS 


15 


9 


9 


6 


13 


2 


iH 


8 


51 


25 
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RESULTS 



Three types of analysis were performed upon the data. Analyses of 
covariance were employed to test for the significance of differences 
between each of the groups (treatment differences). In addition, T tests 
were used to determine the significance of differences within a given 
group from one trial to the next (training differences). Finally, a 
multiple regression analysis was performed on all of the major response 
categories to determine significant relationships among these teach- 
er-pupil behaviors. 

Treatment Differences ; Positive teacher reinforcement constitutes the 
major response category of the dependent variable. Using trial one scores 
as covariants, the groups were found to be significantly different from 
each other. The data in Table 5 summarize these results, and show that 
the differences were significant for both positive verbal (p=.001, .005 
and .025 for trials 2, 3, and 4 and non-verbal reinforcement (p=.025 
and .005 for trials 2 and 4). (See Table 5, Experiment I) 

When the two types of positive reinforcement are taken together, it 
can be seen that the R + D group outperformed all other groups. These 
relationships are illustrated in Figure I. (See Figure I, Experiment I) 
Positive verbal and non-verbal teacher reinforcements are presented in 
terms of adjusted mean frequencies for all trials and groups. Figure 
2 presents the change for positive verbal and non-verbal separately. 

Negative verbal and non-verbal teacher reinforcement occurred infre- 
quently throughout the groups. Table 6 shows that all three of the experi 
mental groups consistently emitted fewer negatively reinforcing responses. 
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TABLE 5 (EXPERIMENT I) 

SUl-lMARY OF THE ANALYSES OF COVARIANCE 
FOR THE EXPERIMENTAL GROUPS, WITH TRIAL ONE SCORES 
AS COVARIANTS AND TEACHER POSITIVE VERBAL AND NON-VERBAL 
REINFORCEMENTS AS THE DEPENDENT VARIABLES 





Group 1 

(c) 


Group 2 
(S-F) 


Group 3 
(R) 


Group 4 
(R+D) 


df 


F 




Adjusted Means and 


Standard 


Errors 






Verbal 
Trial 2 


30.51 

H.H 6 T 8 


30.18 

H.3I6O 


45.18 

4.2841 


60.93 

4.2532 


3/60 


11 . 884 *** 


Non-Verbal 


11.50 

3.1517 


9.39 

3.0566 


i 13.54 

2.9530 


22.13 

2.9556 


3/60 


3 . 482 * 


Verbal 
Trial 3 


18.. 6l 
H .231 


28.83 

4 . 3^+2 


33.46 
4 .. 272 


50.66 

4.243 


3/62 


12 .i 4 o*** 


Non— V 6 rb£tl 


13.97 

2.8720 


7.12 

3.9639 


12.40 

2.8627 


16.. 62 
2.86 8 


3/62 


1.758 


Verbal 
Trial U 


27.00 

5.319 


23.51 

5.283 


26.06 

6.323 


47.. 21 
5.146 


3/52 


4.318* 


Non-Verbal 


6 .. 67 
2.30U1 


7.90 

2,2954 


9.51 

2,6702 


19.28 

2.2145 


3/52 


6,561*** 




Frec^ucnci^ oP positive reinPorcement 




Figure 1, Experiment I 

Changes in Frequency of Positive Verbal and of 
Non-Verbal Combined for Each Experimental Group 



o 

-D 

i- 

> 

+• 






00 



'a 




r: 


1 

OO 




a: 


o 






CM 

SJ 




to 


i: 


u 

O 


U: 

o 

a 


O 


O 

• 




1 


■n 




o 


CM 


, 


i: 


c 


O 


o 




Ok 

c 

o 

iD 

oo 

c 

s 

o 

0 



fO 



CM 



Is- s9 u^ ^0 CM ^ Y 

4 ,u^iuoojoj\j^\oj OAi^isod Jo hou^nbojj 



^ o 



OJ K) 



4J 

s 

5 

M 

O 

a 

»< 



CM 

O 

M 

3 

60 



Changes in Frequency of Positive Verbal and 
Positive Non-Verbal Reinforcements for Each 
Experimental Group. 



over trials while control group responses increased. These differences 
appear to be fairly stable, but do not reach an acceptable level of sig- 
nificance. (See Table 6, Experiment I) Figure 3 illustrates group trends 
in negative reinforcement. The data here are also based on adjusted 
treatment means. (See Figure 3, Experiment I) 

Training Differences ; Table 7 summarizes the significance levels obtained 
when within-group treatment means were compared using the T statistic 
(Winer, 1962, pp. 55-60). As can be seen. Group (R + D) interns signifi- 
cantly increased their rate of positive reinforcement by trial 2 (p=.01). 
(See Table 7, Experiment I) They increased from a base rate of reinforcing 
approximately 60% of all PPR’s to a rate of 76% by trial 2. In trial 4 
the rate dropped to 67% and this combined wit' considerable variation 
within the group, produced before and after treatment differences that 
were non-significant (p=.10). However, when trial 1 versus trial 4 
differences were tested on the assumption that the population variances 
were unequal, significance well beyond the .05 level was obtained 
(required .05 = 2.13j obtained T -- 4.19, Ferguson, 1959, pp. 143-145). 

Group (R + D) subjects also increased their rate of positive non-verbal 
reinforcement from trial 1 to 2 and following two treatment sessions, 
ended (p=.10) to use less negative verbal reinforcement. 

Increases in mean positive verbal reinforcement and a concomitant 
drop in negative reinforcement can be most clearly seen in Group (R) 
subjects. The higher significance levels result from considerably less 
within-group variation. 

The control group showed no significant within-group shifts in 
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TABLE 6 (EXPERIMENT I) 



SUMMARY OF THE ANALYSES OF COVARIANCE FOR THE EXPERIMENTAL GROUPS, 
WITH TRIAL ONE SCORES AS COVARIANTS AND TEACHER NEGATIVE VERBAL AND 
NON-VERBAL REINFORCEMENTS AS THE DEPENDENT VARIABLE 





Group 1 

(c) 


Group 2 
(S-F) 


Group 3 
(R) 


Group 4 
(R+D) 


df 






Adjusted Means and Standard 


. Errors 


F 


Verbal 
Trial 2 


5.15 

1.4i49 


1.88 

1.3759 


5.53 

1.3392 


5.52 

1.3287 


3/60 


1.663®" 


Non-Verbal 


1.7069 

4.695 


1.0716 
4 . 863 


.5838 

4.701 


1.6419 

4.703 


3/62 


1.264 


Verbal 
Trial 3 


4.t4 

1.525T 


5.37 

1.4327 


3.87 

1.3940 


6 . 06 
1.3838 


3/59 


0.446 


Non-Verbal 


1.51 

0.5384 


1.07 

0.5054 


0.58 

0.4887 


1.81 

0.4899 


3/59 


1.184 


Verbal 
Trial 4 


5.TT 

1.3748 


3.39 

1.3790 


2.61 
1 .6l64 


3.29 

1.3304 


3/52 


.941 


Non-Verbal 


1.5T 

0.5060 


0.94 

0.5127 


0.84 

0.5491 


0.75 

0.4623 


3/52 


0.535 



^ .250 level of significance 
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Changes in Frequency of Negative Verbal 
and Non-Verbal Reinforcements for Each 
Experimental Group. 



t:ABLE 7 (EXPERIMENT I) 



SIGNIFICANCE OF DIFFERENCES FOR THE FOUR EXPERIMENTAL GROUPS 
FROM TRIAL TO TRIAL, ON MAJOR RESPONSE CATEGORIES 
OF THE DEPENDENT VARIABLE* 



Response I 
Category 


Group 1 (Control) 


Group 2 ( S-F ) 


Group 3 (P) 


Group H (R+D) 




Direction of Difference from Trial to Trii 
Level of Significance 


il and 


+VR 


NS 


NS 


T^<Tg (.01) 

Ti^<T 2 (. 05 ) 


T^<Tg (.01) 
(.10) 


+NVR 


NS 


NS 


NS 


T^<T2 (.10) 


-VR 


NS 


NS 


NS 


NS 


-NVR 


NS 


T^>Ts (.10) 

T^>T^ (.10) 


Tj^<T^ (.05) 

T^<T^ (. 05 ) 


(.10) 



*Signif icance levels were tested by the T statistic for ccinparisons 
among treatment means (Winer, 19^2, pp. 65 -TO)- 
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Total Pup'll Responses 




(SrF) 



^ ^ L 

2 3 4 

Teaching session 

Figure A, Experiment I 

Changes in Mean Number of Total Pupil Responses 
for Each Experimental Group. 

(Scores adjusted to a mean of zero) 



reinforcement from trial 1 to trial 4. Note that the control group 



subjects tended to increase their negative verbal reinforcement rate 
throughout treatment. 

Pupil Responses ; What are the probable effects of the above types of 
teacher training on pupil behavior? To answer this question, total pupil 
responses and relevant component responses were analyzed ^ A summary of 
the analyses of covariance (Table 8) performed on total pupil responses 
with trial 1 scores as covariates, shows that shifts in pupil responses 
closely followed concomitant shifts in teacher positive reinforcement- 
(See Table 8, Experiment I) while the increase in the Group (R) pupil 
sample appears short-lived, Group (R + D) pupils maintain significantly 
higher response levels in trial 4 (p=.005) as well as for trials 2 
(p=.001) and 3 (p=.01). These data are illustrated in Figure 4. (See 
Figure 4, Experiment I) Note that while the control group initially 
showed a higher mean frequency of responses it dropped slightly over 
four trials. Group (R) and (R + D) pupils increased from trial 1 to 
trial 4. 

The increase in total pupil responses immediately leads one to ask 
whether they are due to increased teacher positive reinforcement, or 
perhaps more simply, to increased questioning by the teacher. 

While it is clear that differential feedback and reinforcement affected 
teacher behavior, it does not necessarily follow that increased positive 
reinforcement as defined here had an effect on pupil behavior . A com- 
parison of volunteered pupil responses as opposed to teacher-solicited 
pupil responses is relevant. If increased pupil responses were largely 
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TABLE 8 (EXPERIMENT I) 



SUMMARY OF THE ANALYSES OF COVARIANCE FOR THE EXPERIMENTAL GROUPS, 
WITH TRIAL ONE SCORES AS COVARIANTS AND TOTAL STUDENT 
RESPONSES AS THE DEPENDENT VARIABLE 





Group 1 

(c) 


Group 2 
(S-F) 


Group 3 
(R) 


Group 1 * 
(E+D) 


df 


F 


Adjusted Means and Standard Errors 


Trial 2 


72.59 

5 . 5351 * 


1*7.39 

5.2296 


65.93 

5.3011* 


83.29 

5.1161 


3/60 


8.267*** 


Trial 3 


58.97 

6.6169 


5H.26 

6.0530 


60.06 

6.3019 


8 H .31 

5.9179 


3/62 


5 . 082 ** 


Trial H 


56.92 

6.6OH 


1*1*. 91* 
6.517 


1 * 7.82 

7.752 


TH .12 

6.309 


3/52 


k.OOh* 



* .05 level of significance 

** .01 level of significance 

*** .001 level of significance 



due to increased questioning by the teacher then one would detect an 
increase in directly solicited answers, and a decrease or no change in 
pupi l”Volunteered statements and questions. This does not appear to be 
the case. F ratios based on covariance analyses of Pupil-Volunteered 
Statements (Table 9, Experiment I) were significant for trials 2 (p*.01) 
and 4 (p=.01). Group (R + D) subjects after showing sharp gains dropped 
off somewhat in trial 3, but rose again in trial 4. Note however, that 
control group in frequencies followed a similar pattern while those for 
groups (R) and (S-F) moved in opposite directions. Figure 5 illustrates 

these trends. (See Figure 5, Experiment I). 

Figure 6 illustrates the relationships between teacher-specified 
responses and pupil responses in relation to ill pupil responses. It is 
a graphical representation of the relevant correlation coefficients re- 
ported in the next section. Mean treatment frequencies of those responses 
directly solicited by the teacher (DSI and DSG-1) and pupil-volunteered 
statements and questions (V and V?) were summed for each trial. This sum 
was then divided by total pupil -volunteered responses (V + V?) so that the 
proportion of pupil-volunteered responses to teacher-solicited responses 
could be determined for each trial. ^ Note that from trials 1 to 4, an 
increasing percentage of volunteered responses contributes to the combined 

(V + V?) 

(DSI + DSG-1) + (V + V?) 



1 



TABLE 9 (EXPERIMENT I) 



SUMMARY OF THE ANALYSES OF SOVARIANCE FOR THE EXPERIMENTAL GROUPS, 
WITH TRIAL ONE SCORES AS COVARIANTS AND TOTAL STUDENT VOLUNTARY 
STATEMENTS AS THE DEPENDENT VARIABLE 





Group 1 

(c) 


Group 2 
(S-F) 


Group 3 
(R) 


Group 1 
(R+D) 


df 


F 


Adjusted Means and Standard Errors 


Trial 2 


13.23 

2.H159 


5.98 

2.3169 


12.51 

2.293H 


17.31 

2.2H22 


3/60 


H,l 8 H** 


Trial 3 


10 . iH 
2.3562 


8.20 

2.1965 


10.95 

2.1850 


15.10 

2.1588 


3/59 


1 -T 86 ®' 


Trial H 


12.27 

2.3098 


7.30 

2.3056 


8.80 

2.5336 


17. Hi 
2.0725 


3 /W 


H. 159 ** 



250 level of significance 
** .01 level of significance 
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Fre<\uency of volunteered responses 




Figure 5, Experiment I 



Changes in Frequency of Pupil Volunteered Responses 
Across Teaching Sessions for Each Experimental Group. 
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2 3 

Teaching sessions 




Figure 6, Experiment I 

Proportion of Pupil Initiated Response in Relation to 
Combined Teacher and Pupil Initiated Responses for Each 
Experimental Group Across Teaching Sessions. 
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Feacher specifies response • 0---0 

Pupil volunteers response • o 



12 3 4 

Teachinc] sessions 

Figure 7, Experiment I 







Adjusted Mean Frequencies of Teacher-Specified 
and Pupil-Volunteered Responses for Maximum 
Treatment (R+D) and Control Croups (C) . 



totals in Groups (R) and (R + D) while the control group shows smaller 
increases. Self-feedback subjects show a distinctly different pattern. 
This decrement is consistent with their performance in terms of 
teacher reinforcement. 

In Figure 7, teacher-specified (DSI + DSG-1) and pupil-volunteered 
responses (V + V?) are presented for the controls and Group (R + D) . 

The frequencies plotted are based on unadjusted treatment means. 

Finally, it is of interest to consider intercorrelations between 
various response categories of the dependent variable, and total pupil 
responses. The intercorrelation matrix for these relationships is pre- 
sented in Table 10. (See Table 10, Experiment I). These correlational 
data are consistent with the earlier reported results. Teacher positive 
reinforcement was found to be significantly related to total pupil 
responses (.50) and to volunteered pupil statements (0.46). Teacher- 
specified questions were also significantly related to total pupil 
responses (0.56). However, when they are considered in relation to 
volunteered pupil responses, the relationship is very slight (+0.07), 
and negative (-0.13) when we consider volunteered questions in relation 
to teacher-specified questions which are directed to the class as a whole. 

An initially surprising relationship obtains between total pupil 
responses and negative verbal reinforcement by the teacher (+0.45). 

One interpretation would be that negative verbal reinforcement pro- 
duces pupil attention. In addition, it also prob*ably has feedback- 
value--it will be recalled that "No" and "Wrong" responses by the 
teacher were included in the negative reinforcement response category. 



TABLE 10 
(Experiment I) 
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Discussion of Results 



It is clear that feedback conditions proved to be the most effect- 
ive training arrangement. Perhaps of greatest interest are what appear 
to be those variations in feedback which are most effective. Clearly, 
adding cue discrimination to the training method substantially improves 
the procedure. This procedure, however, is the most "costly" in that 
it requires the active involvement of the experimenter to describe 
salient cues and to suggest ways of reinforcing participating behavior 
that the subject could use. 

Variations in the effectiveness of this procedure might occur when 
a variety of experimenters are used. Informal observations suggest that 
not all trainees responded equally positively to this condition. Trainee., 
characteristics probably interact with experimenter characteristics, 
and such interactions probably influence differentially the effects of 
feedback and cue- discrimination training. 

Equally interesting is the relative ineffectiveness of the self- 
feedback condition, attractive because it is the least costly procedure. 
This method is probably ineffective because the desired response is no t 
adequately cued. Even if trainees had a limited response repertoire of 
reinforcing responses, they still could have used them consistently, 
if somewhat monotonously. But, the rate increase is not likely to 
occur if the trainee does not "know" when to emit the desired response. 
Both of the other feedback conditions cue as to appropriate response 
in some form. 

This self-feedback condition might be improved by introducing some 
cueing procedures. Or, a combination of viewing models and self might be 
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effective. The results obtained in this experiment indicate only 
that a limited kind of self -viewing , presumed to be a self-feedback 
condition and designed to be so, is not highly effective in pro- 
ducing behavior change. 

The results of this experiment suggest that the operant condi- 
tioning model may be extended to situations in which the learner is 
not actually behaving but merely watching his performance after the 
actions have occurred „ This extrapolation, if further substantiated „ 
greatly increases the application of this particular paradigm. How- 
ever, further research must also be directed to an analysis of the 
viewing conditions--characteristics of the persons viewing, time in- 
terval between enacting and viewing, the kind of behavior being re- 
inforced, and similar conditions which might reasonably be expected 
to enhance or to limit the effects of the reinforcement procedure. 

Also of theoretical interest is the possibility of modifying the 
operant conditioning paradigm by instructing subjects. This procedure 
shortens the time and cost of shaping the desired behavior through a 
series of successive approximations. The learning paradigm of this 
arrangement needs explication and analysis. 



- 58 - 



CHAPTER III 



EXPERIMENT II 

EFFECTS OF FEEDBACK AND PRACTICE CONDITIONS 
ON THE ACQUISITION OF A TEACHING STRATEGY 



Material for this study may be found in Appendices G and H. 



1 




A key problem in the development of instructional systems designed 
to produce teaching skills is that of providing adequate feedback on 
the teacher's performance. Quite apart from the practical limitations 
encountered in trying to schedule immediate feedback sessions with 
teacher trainees, supervision tends to focus on the end products of 
performance rather than the course of learning. This is simply due to 
the fact that it is not usually possible to provide feedback during 
the teaching performance. Discrimination training is necessarily 
based on the perceptions of the teacher and the supervisor as they 
remember the lesson. The entire process thus tends to invite a heavy 
reliance on private frames of reference. The supervisor and the teach- 
er do not start from a common perception of what was done, how it was 
done, and what the effects were. 

Perhaps the signal advantage of televising trainee lessons is that 



in subsequent supervision sessions, the original performance can be com 
pletely reinstated. In this way, the teacher intern is not required to 
respond to supervision on the basis of what he and the supervisor re- 
call about a complex series of events. Further, the usual inhibitory 
effects of delayed feedback may be offset, not only because the initial 
teaching performance is reinstated, but because it is possible to pro- 
vide discrimination training at any point in the development of a re- 
sponse sequence. Rather than reacting to performance in terms of end 



products, it is possible for the intern and the supervisor to analyze 
a given interaction sequence in the classroom; identify salient cues; 
develop strategies to improve further performance. The results of the 
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experiment reported in the preceding chapter support these hypotheses. 
Discrimination training during videotaped playbacks of intern lessons 
produced significantly greater increases in selected teacher behaviors 
than did confirmation or self-feedback procedures. 

If the above reasoning is valid, the reinstatement of original per- 
formance by videotape does in fact solve the problem produced by delay- 
ing feedback and interspersed practice sessions take on new meaning. 

For example, given the capability to reinstate the original performance 
by videotape at any point in time, should practice precede or follow 
supervision? In the first case, the acquisition of a complex skill 
might be slower. However, substantial gains might be realized because 
the learner becomes less defensive about feedback based on his earlier 
performance, and thus becomes more responsive to supervision. This 
would be "delayed" feedback in the sense that the learner was receiving 
training based on an earlier performance. In this kind of a training 
situation, the learner would be serving as his own model. 

Where supervision immediately followed performance, the learner's 
ego -involvement in the lesson just taught might lead him to be less 
open to suggested change. It is also probable, of course, that the 
immediacy of supervision would outweigh such potential resistance. 

There is a general consensus that initially at least, massed practice 
is optimal for the acquisition of complex skills and reasoning strategies. 
Given time limitations in training, however, we are still left with the 
problem of retention. In the study described in the preceding chapter. 
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it was found that while interns quickly reached criterion (where the de- 
pendent variable was amount of reinforcing of pupil participation) initial 
gains were not lasting. This result suggests that distributed practice 
would be a logical alternative. Further, given the perceptual adequacy 
of videotaped feedback, it is quite possible that acquisition rates 
would not be significantly lower than under massed practice conditions. 

Finally, while there is considerable evidence to suggest that 
distributed practice is generally superior to massed practice, why 
this is so is not known. However, as Hilgard (1962) points out, 
none of the current theories account for exceptions to the general 
rule. In training, then, the optimal organization of practice 
periods becomes an inelegant process of empirically establishing the 
limits of each approach in terms of a given skill or strategy. The 
purpose of the experiment reported here was to compare several methods 
of distributing practice and feedback when the latter employed video- 
taped performances of the learner. 

General Procedure ; The general procedures followed in this research 
were similar to those reported in the first experiment. Intern teach- 
ers were videotaped on four separate occasions, during the first 20 
minutes of regular classroom lessons. In the intervals between each 
of the taping sessions, they received experimental treatments within 
the context of regular supervision. 

Supervisory sessions were alike for all subjects who viewed video- 
taped playbacks of their earlier teaching performance with an experimenter 
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who provided discrimination training. However, instead of varying the 
amount of feedback within each session as in the first experiment, we 
held within-session feedback constant, and manipulated the amount of 
practice and delayed feedback over four experimental groups. In addi- 
tion, the posttest was videotaped approximately seven weeks after train- 
ing so as to permit inferences about retention as well as acquisition 
curves . 

The Dependent Variable ; In the earlier experiment, interns had been 
trained to reinforce pupil participation in discussion and review 
lessons. A natural extension of this skill seemed to be those techniques 
where the teacher could increase the quality of such participation. 

Rather than attempting to operationally define a "penetrating ques- 
tion", or to develop techniques for suppressing superficial "first- 
answer" pupil responses, the approach taken was to develop classroom 
techniques which followed simple shaping procedures. Interns were 
given three basic rules: 1) Do not give immediate answers to pupil 

questions. 2) Once a pupil has responded, try to get him to "go be- 
yond" the information given (by one of the several specified techniques), 
3) Differentially reinforce pupil responses that demonstrate increased 
critical awareness. 

In the initial written instructions to each intern, and in super- 
vision as well, a series of discrete techniques were presented following 
the statement of each rule. The basic strategy in each case was the same. 
Following a pupil response, the teacher asked a question designed to 
elicit more information or more meaning from the pupil. If the pupil 
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response was adequate or insightful, the teacher then attempted to get 
him to relate the questions, answer, or comment to another area, or 
to spell out its implications for a given issue. 

These various techniques were presented as exemplars of a basic 
questioning strategy termed probing . Specific techniques were defined 
as; clarification , critical awareness , redirection , prompting and re- 
focus . (The term used to describe each specific category reflects the 
teacher's goal when using a given technique). 

In addition, a secondary set of techniques designed to achieve the 
same ends as probing procedures were included as part of the training 
problem. 

The latter techniques differ from probing in that they may or may 
not depend on a prior pupil response, i.e., the teacher can introduce 
them at any point in the discussion himself, or use them as specific 
types of probes. These techniques were termed encour aging alternatives 
(divergent thinking), supposition (role play in brief) and pupil suit^ 
mary of discussion. Both in training and in later tape analysis, the 
pupil response served as an S to cue the intern or rater that a teach- 
er probing (classifiable) response should immediately follow. 

Treatment Conditions: Stanford Interns were assigned to one of four 

experimental groups, each of which received varying amounts of practice 
between feedback sessions. Subjects in each group received discrimin- 
ation training from one of two E's who were present at each feedback 
session. All feedback sessions were thirty minutes in length. A sum- 
mary of the design appears in Table 1. (See Table I, Experiment II). 
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TABLE 1 (EXPERIMENT II) 



SUMMARY OF THE EXPERIMENTAL DESIGN 



Practice Conditions 


Feedback on Teaching Performance 


1 

Immediate 


Delayed 


Massed Practice 


Group 1 (N = 21] 

i 




Distributed Practice (1 w^iek , . 

intervals) 


. Group 2 (N = 21] 


Group 3 (N * 2; 


Distributed Practice (2 w$ek , . 

intervals) 




Group 4 (N = 22 



TABLE 2 (EXPERIMENT II) 
VIDEOTAPING AND FEEDBACK SCHEDULE 



Pretest Trials Posttest 







)ay 1 


Day 2 


Day 3 


Day 4 








Original Tape 


1 


2 


3 


45 


Massed Practice 


GROUP 


I 


Feedback 

Session 


1 


2 


3 


(45) 


Immediate Feed,- 
back 




1 


Original Tape 


1 


7 


14 


45 


Distributed . 

Practice 


GROUP 


II ; 

1 

1 

J 

- 1 


■ Feedback 
Session 


1 


7 


14 


(45) 


Immediate ,, , 

Feedback 




1 


■Original Tape 


1 


7 


14 


45 


Distributed . 

Practice 


GROUP 


III 


. Feedback 


7 


14 


21 


(45) 


Delayed^ ,, , 

Feedback 






' Session 

i 














f 

1 Original Tape 


1 


14 ^ 


28 


45 


Distributed . 

Practice 


GROUP 


IV 


Feedback 

Session 


7 


21 


35 


(45) 


De ^®y®^pgg^jback 



Group 1. (Immediate Feedback -- Massed Practice ); Group 1 subjects, 
like those in the other three groups, initially received written 
instructions on probing. The instructions related the new techniques 
to the interns' prior training in reinforcement, and stated that the 
basic aim of the current study was to help interns develop a broader 
range of basic questioning skills in discussion and review lessons. 

Each rule was then stated, and specific techniques together with rele- 
vant examples were presented. One of the two E's then briefly reviewed 
the techniques with the intern, reinforced positive statements about the 
potential utility of probing techniques, and viewed intern's playback 
with him. 

Each time a pupil responded verbally, E cued the intern, and if 
the latter had probed, then E reinforced him. If he had not, E suggested 
how this might be done. Interns in the first group were both video- 
taped at the school on a given day, and received supervision based on 
that performance the same evening. The posttest was videotaped 45 
days after the pretest was made. (See Table 2 for videotaping schedule.) 
Group 2 (Immediate Feedback - Distributed Practice ); These subjects 
received the same kind of discrimination training as those in Group 1 . 
Instead of receiving massed practice however, they were directed to 
practice probing techniques for a one-week period following each of 
the supervision sessions. 

Supervision sessions were based on videotapes that had been re- 
corded earlier on the same day. The posttest followed the pretest 
videotape by 45 days. 
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Group 3 (Delayed Feedback -- Distributed Practice ); Subjects in this 
group received the same supervision treatment as those in Group 2. 

However, they always viewed a videotape of their performance which had 
been filmed one week earlier. Unlike Group 1 and 2 subjects who received 
feedback based on that day’s performance, these subjects were taped 
on the day following supervision, and received no discrimination 
training based on their performance for one week. At the conclusion 
of each session. Group 3 subjects were directed to practice probing 
technic^ues not only in the next day’s lesson but during the rest of 
the week as well. 

Group 4 (Reinstated Feedback ■■ Distributed Practice ;) These subjects 
received treatment distributed over a six-week period. Following taping 
on the first day of week one, they received supervision based on that 
tape on the first day of week two. They were taped and supervised on 
alternate weeks. This group was also posttested 45 days after the 
pretest . 

Sub iec ts ; Interns were selected from the same sample as that for the 
first experiment.^ A total of 85 intern teachers majoring in English, 
social studies, mathematics or science were selected for study. Approx- 
imately equal numbers of interns from each subject-matter area, and 
from socio-economically equivalent schools were assigned to each of the 
four groups. Mean age for the four groups varied from 23.7 to 35.5 years. 
Sex differences and subject-matter in each group were very similar to those 

^There was a three-month break between the conclusion of the first 
study and the beginning of the study being described here. 
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already described in the first experiment. 

Measurement Procedures ; Four videotapes for each intern in each group 
were analyzed by raters trained for this purpose. Four raters received 
intensive training on non-experimental tapes before the analysis of the 
latter was begun. Training consisted of joint rating sessions that 
continued until there was 95% agreement on the major response categories. 

Reliability was maintained by frequent analyses of double-rated 
tapes. In addition, joint-rating sessions were held after each block 
of 50 tapes had been completed so that systematic rater biases could be 
controlled. As rating progressed, it was found that certain tapes 
inevitably yielded low interrater agreement. This occurred when the 
intern in question was unable to maintain classroom discipline, or when 
there was an unusually high rate of interaction combined with "fuzzy" 
audio or visual output. In these cases, the tapes were rated by inde- 
pendent teams of raters -- by dividing the rating task between themselves 
and replaying frequently, two raters were able to record all interac- 
tions. To control for combined rating effects, independent teams were 
used. The identification of these "trouble" tapes posed a problem. 

When the raters were asked to rank the rating difficulty of tapes, 
their judgments were not entirely consistent with reliability checks. 

For this reason, it was finally decided to double-rate all tapes. 

The results of these control procedures are reflected in the relia- 
bility coefficients reported in Table 3. (See Table 3, Experiment II). 
The videotapes on which they are based were selected in the following 
manner. Since coefficients based on all of the tapes would have been 
prohibitively expensive and also time-consuming, 160 tapes were selected 
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TABLE 3 (EXPERIMENT II) 



RELIABILITY COEFFICIENTS BASED ON A RANDOMIZED SAMPLE OF 80 
VIDEOTAPES RATED BY TWO INDEPENDEND OBSERVERS 



RESPONSE CATEGORY 


RELIABILITY COEFFICIENT 


Total Pupil Responses 


O.993H 


Total Teacher Probes 


0.9939 


Total Teacher Non Probes 


0.9871 


Total Teacher Reinforcement 


0.9579 


Probing Sub-Categories: 




a. Clarification 


0.9689 


b. Critical Awareness 


0.8658 


c. Redirect 


0.7633 


do Prompting 


O.591H 


e. Refocus 


0.7312 



using tables of random numbers. Five tapes from each group for each 
trial were randomly selected from pools of 15 to 21 tapes . 1 The co- 
efficients reported then are based on a representative selection of 
independently rated tapes from each group and for each trial. As can 
be seen interrater agreement on the major response categories is very 

high. 

A certain amount of data were lost between initial videotaping and 
statistical analysis. Forty-four tapes were omitted at the outset be- 
cause of technical inadequacy or unduly short tape-time. Fifty-six 
tapes were prorated to bring them up to the 20.0 minute criterion. 

Tapes less than 15 minutes in length were omitted; those between 15 
and 20 minutes were prorated. Before any of the above adjustments were 
made, the T statistic was applied to mean tape-time and number of 
omitted tapes per cell to determine whether or not any significant 
differences between cells existed. The results were well short of 
significance as these tapes were almost equally divided among all cells 
The standard errors for short tapes varied from .40 to 1.2 minutes. 

As a further check on the data, the mean number of days between pre- 
and post-test were calculated for each experimental group. The over- 
all mean was 45.25 days, with Sg = 1.41 days. 

RESULTS 

Probing was presented as a basic questioning strategy in training, 
and specific techniques such as Refocus o r Critical Awareness w ere 
subsumed under this broad general concept. It is important to note 

^Team ratings were not included in selection because they had 
rated by special techniques, and were known to be highly reliable on 
all major response categories. 
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that this was a logical distinction made to facilitate training. It 
does not follow that it is psychometrically relevant. In fact, the 
analysis of results did not proceed in terms of one broad dependent 
variable, but in terms of eight. Each of the questioning and related 
techniques were viewed as discrete dependent variables. Table 4 pro- 
vides support for this procedure. The intercorrelations among the 
subcategories of probing are low. Of the 15 intercorrelations, 5 are 
significantly different from zero. Of these, 3 reach the .05 level 
and 2 reach the .01 level. Prompting is significantly related to re- 
direct (r ■ .24, p ■ .05), refocus (r ■ .28, p ^ .05) and clarification 
(r ■ .36, p^ .01). However, prompting occurred infrequently (group 
means varied from 3.7 to 1.7) and contributed little to overall differ- 
ences. Each of the subcategories correlates significantly with probes. 

In the analysis of treatment differences then, F ratios for all of 
the major probing techniques must be considered. F ratios based on 
probing provide a general overview of the results taken as a whole, but 
since they include variables which did not reflect significant differ- 
ences, there is a suppression effect. 

Treatment Differences ; Analysis of covariance with relevant trial 1 
scores as covariates were carried out to determine between-group differ- 
ences in trials 2, 3, and 4. Table 5 summarizes these results. As can 
be seen, differences are reflected in several of the dependent variables.^ 
1 

In general, the most frequently occurring responses are listed first 
in the tables. Further, the more frequently a response occurs, the 
more highly correlated it is with probing. 
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NOTE: In order for any of the coefficients reported above to be considered significantly different 

from 0, r must — .232 (p = .05) or .303 (p = .01). 



1 






TABLE 5 (EXPERIMENT II) 

SUMMARY OF F RATIOS AND SIGNIFICANCE LEVELS FOR THE 
DEPENDENT VARIABLES, DERIVED FROM TRIAL TWO, THREE AND FOUR 
ANALYSES OF COVARIANCE, WITH TRIAL ONE SCORES AS COVARIANTS 



Response 


Trial 2 


Trial 3 


Trial 4 




Category 


F Ratio 


df 


F Ratio 


df 


F Ratio 


df 


Probing 


0.560 


3/58 


1.100 


3/53 


.053 


3/52 


Clarification 


1.170 


3/58 


0.505 


3/53 


.200 


3/52 


Critical 

Awareness 


0.990 


3/58 


1.887® 


3/53 


.280 


3/52 


Redirection 


3.476* 


3/58 


2.631® 


3/53 


.068 


3/52 


Proinpting 


1.147 


3/58 


0.746 


3/53 


.620 


3/52 


Refocus 


1.107 


3/58 


2.822^ 


3/53 


3.134* 


3/52 


Encouraging 

Alternatives 


0.370 


3/58 


1.743a 


3/53 


.516 


3/52 


Pupil Summary 


1.122 


3/58 


.500 


3/53 


.792 


3/52 


Role Play 


0.595 


3/58 


.457 


3/53 


1 . 218 a 


3/52 




N » 63 




N » 74 




N « 53 


i < 




TABLE 6 (EXPERIMENT II) 



SUMMARY OF THE ANALYSES OF COVARIANCE FOR THE EXPERIMENTAL 
GROUPS, WITH TRIAL ONE SCORES AS COVARIANTS AND PROBES 

AS THE DEPENDENT VARIABIE 













Group 1 Group 2 Group 3 Group 4 


df 


F 


Adjusted Means and Standard Errors 




Trial 

2 


12.68 

M 


8.34 


15.07 


11.54 


3/58 


0.56 


o 3.94 
Se 


3.61 


3.95 


4.56 




Trial 

3 


^ 10.20 
M 


5.01 


12.30 


5.54 


3/53 


1.10 


se 


3.28 


3.28 


3.97 


Trial 

4 


M 


7.60 


6.05 


7.84 


3/52 


.05 


Se 


3.31 


3.65 


4.12 
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Only one significant treatment difference, refocus (p^ *05), 
appeared in trial 3, though some differences approached significance 
(p ■ .25) on the variables, critical awareness , redirection , and 
couraging alternatives .^ However, the relative frequency of occurrences 
for each of the response categories that determine treatment differences 
is important in attaching meaning to these results. For example, role 
play occurs infrequently. Clarification , which contributes heavily to 
total teacher responses, occurs frequently. 

Treatment differences in trials 2 and 4 are much less general even 
when those approaching significance are included. In trial 1, redirec- 
tion was significant (p<.05). The F ratio for probing is only 0.600. 

In trial 4 refocus shows treatment differences (p^.05). 

An overall summary of treatment differences for probes derived 
from the analysis of covariance is shown in Table 6. (See Table 6, 
Experiment II). Adjusted means, standard errors and F ratios for mean 
frequencies of probes by groups over trials are presented. Figures 1 
and 2 (See Figures 1 and 2, Experiment II) illustrate the general pat- 
tern of probing . and clarify specific between-group differences for a 
given trial. Both figures are based on adjusted means so that trial 
1 levels of performance can be seen in relation to the other three 
trials. Table 7 shows the proportion each of the subcategories con- 
tributed to the variance in the total probing score. 

1 

Although the .05 level of significance has been used in these studies 
to decide for accepting or rejecting the null hypothesis, when we 
discovered the multivariate nature of the dependent variable, we thought 
calling attention to nearly significant differences would portray the 
results more meaningfully. 
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Figure 2, Experiment II 

Mean Frequency of Total Probes in Proportion 
to Total Teacher Responses Across Teaching 
Sessions for Each Experimental Group. 
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TABLE 7 (EXPERIMENT II) 



PROPORTIONAL OF VARIANCE ATTRIBUTABLE TO EACH 
COMPONENT OF TOTAL PROBING SCORE 





Trial 1 


Trial 2 


Trial 3 


Trial k 


Clarification 


.28 


.26 


.25 


.21 


Critical 

Awareness 


.18 


.17 


.16 


.20 


Redirection 


.15 


.17 


.ik 


.15 


Prompting 


.Ik 


.11 


.13 


.13 


Refocus 


.09 


.10 


.11 


.10 


Encouraging 

Alternatives 


.08 


.07 


.08 


.09 


Pupil Summary 


.08 


.07 


.07 


.06 


Role Play 


.00 


.05 


.06 


.06 



In Figure 1, all unadjusted probing means were brought to a common 
point somewhat analogous to a covariance adjustment by dividing the 
respective means for trial one into each of the four means for each 
group. 

Figure 1 tends to exaggerate certain group trends, particularly 
from trials 3 to 4. Figure 2 is singularly instructive here because 
it considers all of the dependent variables in relation to total 
teacher responses. In this figure plot points were derived by express- 
ing probes as a percentage of total teacher responses. Now, comparing 
Figure 1 with Figure 2, it can be seen that Group 3 and 4 do not drop 
off from trial 3 to trial 4 as implied in Figure 1. Figures 3 and 4 
and 5 illustrate some of the differences across trials in subcategories 
of probing . Recall that clarify and redirect correlate (r = .33), 
while clarify and refocus do not correlate (r ^.21). Refocus and re - 
direct similarly do not correlate. Because the characteristics of the 
dependent variable need further exploration, analyses of differences 
in greater detail did not seem profitable at this time. These graphs 
suggest that the treatments may have affected the subcategories in 
different ways. 
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Figure 3, Experiment II 

Mean Frequency of Clarification Responses Across 
Teaching Sessions for Each Experimental Group. 
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Figure 4, Experiment II 

Mean Frequency of Redirection Responses Across 
Teaching Sessions for Each Experimental Group. 
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Figure 5, Experiment II 



Mean Frequency of Refocus Responses Across Teaching 
Sessions for Each Experimental Group. 



Discussion of Results 



Treatment differences were not entirely consistent throughout 
trials and across dependent variables, so that conclusions must be 
qualified. Essentially no meaningful differences were found. There 
appears to be a differential effect of the treatments on the different 
categories of probing . Since an interaction analysis was not feasible, 
this observation must be accepted as a hypothesis, not a conclusion. 
Although no differences can not be treated as a conclusion, it is in- 
teresting to note that wide variations in time of feedback did not 
produce differences. 

Certain implications can also be drawn about treatment differences 
related to retention. Figure 2 suggests that the distributed practice 
and delayed feedback groups (Groups 2 and 4) maintained relatively 
higher probing response rates on the posttest than did Group I who 
dropped off quite sharply. However, the data do not permit firm con- 
clusions here as only one F ratio for trial 4 was significant. 

Finally, it should be pointed out that there appear to be prac- 
tical limits on the amount of probing possible in any given class 
period. Unlike teacher reinforcement which may occur in high fre- 
quency, classroom and subject matter concerns establish a ceiling for 
probing. The most sensitive measures of this variable are likely to 
be expressed as ratios or percentages related to total teacher re- 
sponses . 
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CHAPTER IV 



EXPERIMENT III 

THE EFFECTS OF MODELING AND 
FEEDBACK VARIABLES ON THE 
ACQUISITION OF A COMPLEX 
TEACHING STRATEGY 



Materials for this study may be found in Appendices G-L 



1 



THE PROBLEM 

The purpose of this study was to assess the relative effects of 
various arrangements of feedback and presentation variables upon the 
performance of a specified class of teacher behaviors. 

A common approach to the transmission of teaching skills has been 
to provide some kind of discrimination training by means of written 
and oral instructions. The intern teacher typically received a des- 
cription of the correct responses and their sequencing for a particular 
situation. He then attempts to produce these behaviors in the class- 
room and receives periodic feedback on his performance. 

^ Another approach to training problems of this sort is suggested by 

recent findings on the role of observational learning in personality 
development. A review of the relevant literature by Bandura and Walters 
(1963b) has shown that complex social behavior may be acquired almost 
entirely through imitation. They state the provision of face-to-face 
models serves to accelerate the learning process and, in cases where 
errors are dangerous or costly, become an essential means of trans- 
mitting behavior patterns (Bandura and Walters, 1963b, p. 52). In 
addition, Bandura, Ross and Ross (1963c) have demonstrated that film- 
mediated models are as effective as real-life models in transmitting 
deviant patterns of behavior. 

I; 

While experimental demonstrations of modeling effects have largely 
employed young children as subjects and aggression as the dependent 



variable, there is little doubt that adult subjects may acquire various 
classes of responses by the same process. The implication of this for 




teacher training is that an audiovisual display of a teacher model 
performing some desired behavior may provide an effective alternative 
to purely descriptive techniques. 

One of the objectives of the study was to compare these two modes 
of presentation in the context of a televised series of training 
sessions for intern teachers. The experimental design permitted an 
analysis of the assumption that the rate and level of learning a given 
teaching skill varies as a function of the mode of model presentation. 
Two types of modeling were considered: 

1. Symbolic Modeling : This is defined as a process whereby one 
transmits desired behaviors to the learner by means of written 
or verbal instructions. The subject does not view an actual 
portrayal of the desired behavior. 

2. Perceptual Modeling : This is defined as a process whereby one 
transmits desired behaviors to the learner by means of a filmed 
model who portrays the desired behavior. 

In addition to presentation variables, the problem of adequate 
feedback on the teacher's performance is of considerable importance in 
the transmission of a teaching skill. The effects of feedback have 
been extensively investigated in terms of reinforcement, knowledge of 
results, confirmation, and trial and error learning. Despite theoret- 
ical debate, there is little doubt that such feedback does produce 
learning (Michael and Maccoby, 1960; Hilgard, 1956). 

While the experimental literature suggests that both modeling and 
feedback effectively change behavior, little is known about the relative 
effectiveness of these two processes. The human learner, because of his 
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considerable information-processing capacities, may require little 
feedback when the desired behaviors are portrayed as in perceptual 
modeling. However, if the responses to be learned are sufficiently 
complex, some combination of demonstration and feedback may be re- 
quired . 

The purpose of the study then was twofold. In addition to per- 
mitting a comparison of symbolic and perceptual modeling, the exper- 
iment was designed to assess the relative effects of various arrange- 
ments of feedback in combination with the two presentation variables. 
In general, the experiment sought to determine which would be more 
efficient; telling the person what to do (symbolic modeling), or 
showing him what to do (perceptual modeling), or some combination 
of these approaches with feedback that includes reinforcement and 
further discrimination training on the relevant cues. 

THE DEPENDENT VARIABLE 

The dependent variable that was developed for the study is termed 
Probing . A complete discussion of the technique appears in Appendix 
G (instructions to Interns) and Appendix H (The Rater's Manual). 

Thus, comments here will be brief. 

Probing is a basic questioning technique in which the teacher re- 
quires students to go beyond first-answer responses. It is designed 
to be used in lessons where pupil participation is prerequisite to 
the goals of instruction, and is intended to upgrade the quality of 

such participation. 

Once the pupil has responded by means of a question, answer or 
comment, the teacher may probe_ this response by means of one or 
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more Probing techniques. These sub-classes or categories of Probing 
are termed: Clarification , Increased Critical Awareness, Refocus, 

Prompting , and Redirect . The labels in each case generally reflect 
the teacher's goal when using a given type of Probing. 

Two basic distinctions have been made about Probing as a dependent 
variable in an earlier paper by Allen, McDonald, and Orme (1966). 

First, each of the response categories of probing acts as a discrete 
dependent variable. This inter-variable independence is shown in Table 
1, Of all the possible intercorrelations between the various response 
categories, only one proved to be significantly different from zero 
(Prompting and Clarification rp . It follows that in the 
analysis for treatment effects, one can expect differential levels of 
significance on different variables in each session. Statistically, 
we are concerned with as many dependent variables as there are sub- 
categories of Probing . In training, these differences were not empha- 
sized to such a degree. Treatment was designed to produce increases 
in Probing as a general questioning strategy. 

The second relevant distinction to be made about Probing is that 
there is a definite ceiling effect on its frequency of occurrence in 
classroom interaction. The teacher can only probe (or non-probe) 
following a pupil response. Since total pupil responses can reasonably 
be expected to vary, to some extent, independently of treatment, 
statistical analysis must go beyond tests of mean differences of 
Probing by group and trial. One must also consider mean Probes in 
relation to Non-probes . These two teacher behaviors when taken together 
yield a frequency which is equivalent to Total Pupil Responses . 
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HYPOTHESES 



1. Presentation Variables 

It was hypothesized that the rate and level of learning a given 
skill varies as a function of the mode of model presentation. 

a) Perceptual modeling procedures will produce significantly 
greater changes in the response strength of desired behaviors 
than will symbolic modeling procedures. 

b) A combination of perceptual and symbolic modeling procedures 
will prove to be more effective than either procedure alone. 

2. Feedback Variables 

a) Prompting feedback will produce greater change than either self 
or confirmation feedback; the latter will produce the most. 

b) A combination of prompting and confirmation feedback will prove 
to be more effective than any of the three forms alone. 

3 . Combinations of Presentation and Feedback Variables 

a) The optimal combination of presentation and feedback procedures 
will be: a) perceptual and S 3 nnbolic modeling in the presenta- 
tion phase of training; and b) prompting and confirmation feed- 
back in the feedback phase. 

b) Initial gains in performance should be greater than in later 
phases of training. It was therefore predicted that perform- 
ance gains following modeling treatments would be significant- 
ly greater than increases in Probing following feedback treat- 
ments . 
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THE DESIGN 



A two by three design matrix that permitted an assessment of the 
relevant combinations of presentation and feedback variables yielded 
six experimental groups. 

A. GENERAL PROCEDURE 

The treatment procedure for each group was broken down into ten 
steps or stages (see Table 2). In six of these steps, all groups 
received identical treatment. In the remaining four, each group was 
exposed to the appropriate type of modeling and a particular type of 
feedback. 

In the first step, all subjects were videotaped while teaching a 
five-minute lesson to four junior high school students. This con- 
stituted the pretest from which baseline levels of probing were derived. 

In step two all subjects received written instructions describing 
the criterion behavior. Following this set-induction process, the groups 
were exposed to the appropriate types of modeling in the next two steps. 
In steps five and six, all subjects planned and then taught a second 
five-minute lesson. The cycle was then repeated except that the set 



^Five minute lessons were employed throughout the study. The rationale 
for this "microteaching" format derives from two considerations. First, 
Margolius and Sheffield (1961) found that in their training film re- 
search, a four to five-minute film segment turned out to be the optim- 
al Demonstration-Attention segment for college-age military subjects. 
Further, McDonald, Allen and Orme (1966) found that if a videotape 
playback ran for twenty minutes, E's comments tended to be perceived 
as repetitious. 

Secondly, the five-minute lessen provided ample opportunity for the 
intern or perceptual model to demonstrate a satisfactorily high number 
of probes for measurement purposes. 
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Feed^acV Modeling 



TABLE 2 



SUMMARY OF STEPS IN TREATMENT BY EXPERIMENTAL GROUP (a) 



Phase and Step in Experimental Group Minutes in 





Treatment 


1 


2 


3 


4 


5 


6 


Treatment 




1. 


Teach (Pretest) 


X 


X 


X 


X 


X 


X 


05 




2. 


Set Induction 


X 


X 


X 


X 


X 


X 


20 


bO C 
C O 
’H S 
-M 


3. 


View The Self 


A 


E 


A 


E 


A 


E 


10 


(U <0 

«o 0) 

O M 


4. 


View the Model 


R 


R 


A 


A 


E 


E 


10 




5. 


Plan Next Lesson 


X 


X 


X 


X 


X 


X 


10 


13 


6. 


Teach 


X 


X 


X 


X 


X 


X 


05 


^ Qi 
(0 B 


7. 


View The Self 


A 


E 


A 


E 


A 


E 


10 


"S CO 

a) 0) 


8. 


View The Model 


R 


R 


A 


A 


E 


E 


10 


lU M 


9. 


Plan Next Lesson 


X 


X 


X 


X 


X 


X 


10 




10. 


Teach (Posttest) 


X 


X 


X 


X 


X 


X 


05 



^Explanation of Symbols: (X) indicates that the subject 
receives this treatment, (a) indicates that the subject views a 
playback of his own performance alone, or else views the perceptual 
model alone; (E) indicates that the subject views the tape concerned 
with the experimenter and thus receives discrimination training and 
reinforcement from him. (R) indicates that the subject restudies 
the written materials describing the criterion behavior . 
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Induction process in step two was dropped. Note that in viewing the 
appropriate model for the second time, the subject was receiving a 
feedback treatment. The second presentation of the perceptual model 
constituted prompting feedback and symbolic modeling was defined as 
confirmation feedback. 

In terms of general procedure then, all subjects were pretested, 
received written instructions, were exposed to the appropriate model- 
ing procedure, and following a planning session, they taught for the 
second time. This means that before his treatment was complete, each 
subject had been exposed to the appropriate modeling and feedback 
treatments, and had taught three times. 

To avoid undue attention being paid to the development of a new 
lesson each time, the subjects taught a different group of students in 
each lesson. This allowed them to retain the same basic subject matter 
in each lesson while attending to improvements in probing techniques,. 

The experiment was run over a six-week period as part of the re- 
gular intern program. Each subject spent approximately one hour and 
a half in total treatment. 

B . TREATMENTS 

As mentioned earlier, six experimental groups received differential 
treatments. Four groups viewed perceptual models at some point in 
their treatments and two received symbolic modeling. 

To facilitate subsequent discussion, the groups will be identified 
by the kind of modeling they initially received. The overall treat- 
ment for each group will be considered in terms of presentation var- 
iables. Following this, feedback procedures will be discussed. 




1 0 Presentation Treatments 

A summary of the following discussion appears in Table 2. 

Group One (Minimal Symbolic Modeling); following the pretest and 
set induction through written materials (Appendix G) , Group One subjects 
viewed a videotape playback of their pretest performance alone. 

Following this they were directed to study the written materials on the 
criterion behavior for a second time., In subsequent steps they planned 
and taught again, then viewed a playback, of their second lesson, re^ 
read the instructions once more and taught the third lesson. 

Note that this is a symbolic modeling group. While, they received 
written instructions , (no verbal instructions), discrimination t.raining 
was provided by E. This group was thus termed a Minima l°S ymbclic Mod- 
eling treatment group. 

Group Two maximal Symbolic Modeling) ; Following the first two steps, 
subjects in this group viewed a videotape playback of their pretest 
performance, with E . As in all treatment steps where he. viewed a tape 
with the subject, E verbally reinforced the. desired responses when they 
occurred during the playback. In addition he identified salient cues 
to which the desired behavior should be attached, made suggestions about 
variations in the form of the desired behavior, and pointed out the 
effects of such behavior on pupil behavior. In short, E's function 
was to provide appropriate reinforcement and discrimination training. 
Following the playback with E. Group Two subjects were directed to 
study the written materials for a second time. Then after a planning 
and teaching session, the cycle was repeated. 
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Note that like Group One, Group Two subjects received symbolic 
modeling. However, in addition to written instructions in step four, 
they also received verbal instruction from E in the form of discrim- 
ination training and reinforcement. This group was therefore termed 
a Maximal Symbolic Modeling treatment group. 

Group Three (Minimal Perceptual Modeling): Following the first 

two steps, subjects in the third group viewed a videotape playback of 
their pretest performance alone . In the next step they viewed a per- 
ceptual model alone . As in all treatment steps where subjects viewed 
a perceptual model. Group Three interns were presented with a model 
teacher who was of the same sex as the subject, and who demonstrated 
Probing in a subject matter area which corresponded to the intern’s 
major field of interest. For example, female English teachers viewed 
a female model teacher who demonstrated Probing in a five-minute 
English lesson. 

Once Group Three subjects had viewed the model alone they planned 
and then taught the second lesson. The cycle was then repeated. Since 
Group Three constituted a perceptual modeling group who viewed the model 
without receiving discrimination training or reinforcement from E, they 
were defined as a Minimal Perceptual Modeling treatment group. 

Group Four (Strong Symbolic, Minimal Perceptual Modeling): Like 

Group Two, this group received both discrimination training and rein- 
forcement from E while viewing videotape playbacks of their own perform- 
ance. This constituted a strong sjnnbolic modeling procedure ( Maximal 
Symbolic Modeling procedures required subjects to restudy the written 
instructions in addition to reinforcement and discrimination training). In 
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the next step, subjects viewed the appropriate perceptual model alone. 
This treatment thus combined Strong Symbolic with Minimal Perceptual 
Modeling . 

Group Five (Maximal Perceptual Modeling): This treatment differed 

from all other in that subjects viewed playbacks of their own perfor- 
mance alone, but viewed the appropriate perceptual model with E. His 
function was to provide discrimination training just as when he viewed 
a subject's playback with him. In this treatment however, his verbal 
output was keyed on the perceptual model's behavior rather than to the 
subject's performance. 

This mode of treatment was termed Maximal Perceptual Modeling because 
in addition to viewing the model, subjects received discrimination train- 
ing from E which was based directly on salient modeling cues, and served 
to increase the distinctiveness of these cues. 

Group Six (Strorg Symbolic and Maximal Perceptual Modeling): Sub- 

jects in the appropriate treatment stages viewed playbacks of their 
own performance with E, and then viewed the appropriate perceptual 
model with E. The condition was thus one of Strong Symbolic and Max - 
imal Perceptual Modeling . 

2. Feedback Treatments 

By definition, all subjects received feedback treatments when they 
were exposed to the appropriate form of modeling for the second time. 

The second presentation of the perceptual model was defined as Prompting 
Feedback , and Symbolic Modeling became a form of Confirmation Feedback . 

A subject who viewed both his own playback and the perceptual model alone 
received Self-Feedback. The experimenter's function, when he viewed a 
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tape with the subject, was precisely the same as in the earlier phases of 
treatment . 

The relationship of each of these forms of feedback to the six ex- 
perimental groups is outlined in Table 3. Groups One and Two did not 
view both the model and their own playbacks, and thus received only 
partial feedback. For this reason they were not considered as one of 
the four basic groups in the analysis of feedback effects. 

C. SUBJECTS 

Intern subjects were drawn from the Stanford Intern Teacher popula- 
tion. Prior to random assignment to the six groups, subjects were 
categorized by subject-matter major. From these subgroups, the interns 
were then assigned to one of the six treatment conditions. Foreign 
language majors were not included in the study, as their tapes would 
have presented undue measurement problems. They also follow a highly 
prescribed methodology that does not permit the systematic use of Prob- 
ing techniques . 

Relevant characteristics of the san^le studied appear in Table 

4. As can be seen, the groups are adequately matched on age, sex, abil- 
ity and subject-matter variables. T tests for differences between groups 
were applied for each of the above variables, and proved to be non-sig- 
nificant. 

Interns and models taught each of their lessons to a "micro-class" 
of four pupils. These students were drawn from the Palo Alto school 
system, and were paid for their services. 

Forty students were hired to work in teams of four. Each team con- 
sisted of two boys and two girls. All pupils were either ninth or tenth 
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TABLE 3 



SUMMARY OF MODELING AND FEEDBACK PHASES 
OF TREATMENT BY EXPERIMENTAL GROUP 



Experimental 

Group 



Phase of Treatment 



Feedback 

Modeling (Steps 7 and 8) 

(Steps 3 and 4) 



1 

2 

3 

4 

5 

6 



Minimal Symbolic Modeling 

SM (min.) 

Maximal Symbolic Modeling 

SM (max. ) 

Minimal Perceptual Modeling Self-Feedback 
PM (min.) 

Strong Symbolic-Minimal 

Confirmation Feedback 

Perceptual Modeling 

SM (Str. ) - PM (min. ) 

Maximal Perceptual Modeling Prompting Feedback 
PM (max.) 

Strong Symbolic-Maximal 

Confirmation and 

Perceptual Modeling Prompting Feedback 

SM (Str. ) - PM (max. ) 



1 




graders, and between 14 and 15 years old. The majority were from middle 
class background. However, approximately twenty percent were of lower- 
middle or upper-lower class extraction. None of the pupils received 
special instructions on how to behave in the microteaching situation. 

They were told only that Intern teachers would be teaching them various 
lessons. No mention was made of Probing . 

D. TRAINING PROCEDURES FOR MODELS 

Experienced teachers were selected to act as perceptual models. One 
model of each sex from each of the major subject matter areas was trained 
to demonstrate Probing techniques in a five-minute lesson. The conditions 
under which both models and interns taught were identical. 

One week prior to taping and training, potential models were given 
an outline of Probing procedures (see Appendix G) . In the training 
session they taught the same lesson to different groups of students 
until criterion was reached. The investigator set a lower limit of ten 
Probes for an appropriate lesson, and in addition required that the 
lesson be of "superior" educational quality in all other respects. 

The latter decision was based on two sets of judgments. At the 
conclusion of each demonstration tape, the students filled out the 
Stanford Teacher Competence Appraisal Guide , an instrument that provides 
student evaluation across thirteen teacher behaviors on an eight-point 
scale (see Appendix K) . In addition, the investigator and one other 
experienced teacher subjectively assessed the general educational quality 
of the tape. The models typically taught the same lesson three times. Dur- 
ing each demonstration, the investigator recorded the number of Probes that 
occurred, and noted suggestions for improvement. In the period between 

-98- 



f 



each tape, the investigator and the model replanned the lesson. In all, 
eleven models received training. From this pool, the investigator se- 
lected the best seven tapes. 

Models *characteristics are summarized in Table 5. It should be 
noted that the same female model was shown to female science and math- 
ematics subjects. The lesson dealt with simple probability notions 
in science and was thus applicable to mathematics. Only three female 
mathematics subjects received perceptual modeling treatments, and since 
the intended female model in mathematics was weak, it was decided to 
forego further training with a new model and to use the science tape. 

Two females in drama, and two in art viewed the female social studies 
model. Four males in physical education viewed the male English model. 
Since there were few subjects in these disciplines, models were not trained 
to demonstrate probing in these areas. The English and social studies 
models were selected in these cases because of all the tapes, their per- 
formances were the strongest in terms of number of Probes demonstrated. 

In sum, eleven subjects viewed perceptual models who demonstrated 
Probing in subject matter areas other than the Intern's major area. 

However, the sex of the model was controlled in all cases. 

E. TRAINING PROCEDURES FOR EXPERIMENTERS 

Two experimenters were used in the study. As pointed out in an 
earlier section, they provided reinforcement and discrimination train- 
ing at appropriate times in the treatments of the various groups. Both 
of the experimenters were male graduate students in Educational Psych- 
ology. Both had had prior experience with conditioning procedures 
and discrimination training in earlier research. 
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TABLE 5 

SALIENT CHARACTERISTICS OF THE PERCEPTUAL MODELS AND THEIR RESPONSE 
FREQUENCIES FOR THE MAJOR RESPONSE CATEGORIES RATED 
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Training was accomplished in the following manner. The author, who 
served as and E 2 jointly studied The Instruction to Interns (Appen- 
dix G) , The Model’s Manual (Appendix I) and The Rater’s Manual (Appen- 
dix H) . Following this and E^ viewed the model tapes. In addition, 

E observed E, modeling relevant discrimination training and reinforce- 

2 ^ 

ment techniques with the first six experimental subjects. Finally, E^ 
observed E 2 working with his first four subjects. As the experiment 
progressed, E-^ and E^ periodically observed each other to ensure that 

they were employing equivalent procedures. 

As each subject completed treatment, he was asked to complete a 
questionnaire which tapped his perceptions of the experimenter's ability, 
sensitivity to Individual differences and general "likeabillty" (Appen- 
dix E) These results provide comparative data on subjects' perceptions 
of the experimenters, and are discussed in the Res ul t ^, section of this 
report. Experimenter effects are also discussed in that section. 

In an attempt to minimize bias, neither experimenter was informed 
of the treatment conditions of the subjects with whom he worked. 

F. METHODS OF GATHERING THE DATA 

During the study, each of the interns' lessons was recorded on 

videotape for analysis. The relevant behaviors were later recorded by 

four raters trained for this purpose. 

Prior to the analysis of the tapes in the current experiment, the 

raters had been trained and had rated approximately 400 twenty-minute 
tapes for another experiment in which the investigator was involved. 
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The dependent variable in this earlier study involved basic questioning 
techniques that included Probing among other things. Thus, the retrain- 
ing phase for the current analysis was relatively brief. The raters re- 
trained on nonexperimental tapes first, then rated the model tapes until 
ninety percent agreement was reached. Retraining included intensive study 
of The Rater's Manual and Coding Form (Appendix H) , and practice on high 
frequency Probing lessons. Total retraining time was approximately six 
hours . 

Once criterion was reached, the raters independently coded the 
experimental tapes. To maintain reliability, all four raters indepen- 
dently rated three experimental tapes out of each block of 50. Analysis 
sessions followed in which degree of agreement was discussed and reruns 
where necessary could be carried out. Note that these tapes were not 
used in the acquisition of reliability data; they served a maintenance 

function only. 

G. INTER-RATER RELIABILITY 

Interrater reliabilities on all types of responses rated proved to 
be very high. For this reason, and because further light is cast upon 
the nature of the dependent variable, the ensuing discussion will be 
fairly detailed. 

The reliability data are based on independent, double ratings of 

all of the videotapes used in the study (N=326) . The original plan, 

as in the Allen, McDonald, Orme (1966)^ experiment employing Probing , 

was to randomly select a representative sample of tapes for double-rating. 

~ ^For ease in exposition, the Allen, McDonald, Orme (1966) 
experiment will hereafter be termed "Experiment 11". 
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However, by the time that the raters had completed coding all of the 
tapes in the earlier study, they had become quick enough to rate a tape 
through with no holdups or reruns. Since the tapes in this study were 
only five minutes long, and since there were too few TV machines for each 
rater to work on a different tape, it was possible to have the tapes 
double-rated without incurring additional expense. Given two playback 
machines, four raters and one operator, it was found that by having the 
raters move from machine to machine (located in different rooms), one op- 
erator was engaged full-time playing tapes, changing them, and cleaning the 
machines. Operators played the tapes in a pre-established order, follow- 
ing a list in which groups and sessions were randomized (randomization 
based on Tables of Random Numbers) . In this way, neither the operator 
nor the rater had any knowledge of the treatment condition or phase in 
treatment of subjects’ tapes. Finally, to avoid the possibility that a dis- 
proportionate number of tapes, albeit independently rated, might be coded 
by the same two raters, a simple schedule of room changes was followed. 

When the data were transferred to IBM cards, ratings on each tape 
were arbitrarily assigned to one of two data decks. Thus there were two 
complete sets of data on all subjects in all groups, the only differences 
between them being that they had been independently rated by different 
raters. Reliability coefficients were determined by running one com- 
plete deck against the other. Coefficients were determined for each of 
the major response categories and sub-categories rated. These data are 
reported in Table 6, along with the reliabilities obtained for the same 
variables in Experiment II ,.(N=79) . 

Clearly, the coefficients are higher than those one has come to 



o 

ERIC 
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TABLE ^ 



INTER-RATER RELIABILITY FOR EXPERIMENT TWO AND THE 
DISSERTATION DATA ON THE MAJOR RESPONSE CATEGORIES RATED 



Response Category 



Ic 


Pupil Questions 


2. 


Pupil Comments 


3. 


Pupil Answers 


Intern Reinforces Pupil Response 


Intern Repeats Pupil Response 


Total Non-Probes 


lo 


Intern Answers Question 


2. 


Intern No Response 


Total : 


Intern Probes 


1. 


Clarify 


2. 


Critical Awareness 


3. 


Redirect 


H. 


Prompt 


5. 


Refocus 


*6. 


Encourages Alternatives 


n. 


Summarizes 


8. 


Role Play 



Experiment II 

(N = T9) 


Dissertation Data 
(N = 326 ) 


.99 


.99 


.73 


.65 


.87 


.88 


.96 


.97 


.96 


.99 


.89 


.98 


.99 


.99 


.62 


.80 


.73 


.91 


.99 


.99 


.97 


.97 


.87 


.9H 


.76 


.97 


.55 


.92 


.57 


.87 


.59 


— — — 


— 


— 



*Response frequencies for these latter variables were too low 
to permit meaningful coefficients. 

^Inter-rater agreement among all four raters on 20 tapes coded 
at different times throughout the sutdy was also determined. It was 
found to be above 90fa on all response categories except for Clarific a 

( 89 ^) and Refocus ( 85 ^) 
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expect on the basis of reading the general literature on rating. It 
should be equally obvious that these results are due to extensive pre- 
training for raters, and unambiguous operational definitions of the var- 
iables involved(c.f Appendix H: Rater's Manual). They are not a func- 
tion of spuriously high agreement that may operate when raters work in 
teams, since they worked independently. It is also highly unlikely that 
pooling effects inflated reliabilities because ratings were not combined 
It is possible that the Total Pupil Responses , Total Prob i t y, and 
Total Non-Probing coefficients are somewhat inflated because they are 
made up of the summation of relevant sub-categories; i e.. 



TOTAL PUPIL RESPONSES + TOTAL PROBES + TOTAL NON-PROBES 



where. Total Pupil Responses 



(Pupil Question) + (Pupil Rhetorical) + 
(Pupil Comments) + (Pupil Answers). 



Total Probes - (Clarification) + (Critical Awareness) + (Redirect) 

+ (Prompting) + (Refocus) + (Encourages Alterna- 
tives) + (Summarizes) + (Role Play) 

Non-Probes * (Intern Answers Pupil Question) + (Intern No- 

Response) + (Intern Reinforces) + (Intern Repeats 

Pupil Response), provided that , in each case, the 
intern then fails to go on and Probe the pupil 
response--the latter qualification being nece- 
ssary so that it would later be possible to deter- 
mine whether a given sub-category in this class 
was employed as a Non-Probe, or used in conjunc- 
tion with Probing^ 



^It should be pointed out here that all of the analyses reported 
in the next chapter were run on one data-deck only. This decision was 
made before reliability data were available. When measures of central 
tendency, distribution and variability were later run on both decks as 
Tcheck! as might be expected from the high reliabilities, differences 
were miniscule. The "second" deck was therefore used only in the 
determination of reliability. 
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The proportionate contribution of each of the subcategories to 
its respective total, and the intercorrelations between them are rele- 
vant in trying to determine whether or not the summated or major response 
categories reflect spuriously high reliabilities. These data are pre- 
sented in Table 7.^ 

In comparing the reliability coefficient for a given variable 
(Table 6) with its proportionate contribution to the relevant summated 
score (say, Total Pupil Responses in Table 7), it will be seen that 
there is a marked tendency for those responses which occurred infre- 
quently to be less reliably rated. For example, Pupil Questions account 

for less than five percent of Total Pupil Responses , and at the same time 

2 

proved to be the least reliably rated of such responses. 

In short, in view of the high reliabilities of the discrete 
response subcategories contributing most heavily to the summated cate- 
gories of Probing , Non-Probing and Total Pupil Responses , it is unlikely 
that the latter are statistically unstable. One ray well argue Instead 
that they could be expected to be more stable than the lesser response 
categories and therefore to be preferred in statistical analysis. 



^Certain other data are reported in Table 7 which are not rele- 
vant to the current discussion -- they will be discussed at a later time. 

2 

Pupil Rhetorical Responses while included in the rating form 
as a possibly meaningful type of response, occurred so infreq^iently that 
analyses on this variable have no meaning. The same holds true for the 
Probing subcategories: Refocus ; Encourages Alternatives ; Summarizes ; 

and Role Play. 
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RESULTS 



A. RATIO AND PROPORTIONATE ANALYSIS 

Ratio analyses were performed on three different sets of scores. 
Probable modeling effects were explored by analyzing Session Two minus 
Session One mean scores (S 2 - and (S^ - scores as well. In 
both cases Pretest or Session One scores were used as covariates. In 
addition, probable feedback effects were analyzed by comparing Session 
Three minus Session Two (S^ - S 2 ) means. In the latter case both 
Session One and Session Two scores were used as covariates, i.e., pre- 
test and modeling effects were employed as covariates so that feedback 
effects could be analyzed apart from these prior influences on perfor- 
mance. The results for all three analyses are presented in summary form 
in Tables 8 and 9. Significance was achieved on the (S 2 - analysis 
(p^.05), but fell short of the .05 level on the (S 3 - S^)analysis 
(p ^.25) where one would most likely expect significance. So also, the 

_ 5 ) analysis yielded an F ratio that was non-significant (p ^^.25) 

3 2 

In the (S« - S,) analysis, differences between Group 4 (SM - strl, 

PM - min) and the lowest scoring group (Group 1-SM min.) accounted for 

the significant F ratio. These results are inconsistent with all other 

results to be reported in two respects. First, differences where they 

occurred were most pronounced for (S^ ” Sj^) analyses, not (S 2 ■ S^^) 

analyses as is the case for the Probe /Non-Probe analysis here. Secondly, 

the magnitude of mean differences in all other analyses led to group 

rankings which are generally consistent with the hypotheses (Group 

6 highest, down through Group 1, the lowest). In the P/NP ratio 
analysis the rank order was instead, 4-2-6-5-1-3. 
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TABLE 9 

SUMMARY OF THE ANALYSES OF VARIANCE FOR ADJUSTED MEAN DIFFERENCES BETWEEN ALL GROUPS 
OVER ALL SESSIONS WITH PROBES OVER NON-PROBES (P/NP) AS THE DEPENDENT VARIABLE 




Similar analyses of Total Probes /Total Pupil Responses were also 

run* Results here were consistent with other analyseSj but fell short 

of significance. However, the (S^ - = 1.797;(p <.. 25 ), and 

(S - s ) ; F * 1.653; (p -<.25) analyses showed a trend in the 

3 2 ' 5.97 

predicted direction. 

In view of the questionable nature of the distributions of ratios 
(P/NP) and proportions (P/TPR) , these results are suspect. Perhaps 
the most meaningful approach to the problem of ceiling effects lies 
in a straightforward though inelegant inspection of the shifts in treat- 
ment means for each group as it moves through the three experimental 
sessions. These shifts are depicted in Figure 1. 

It is clear that the proportion of Probes to Total Pupil Res - 
ponses systematically increases for all groups from Session One to 
Session Two. This is also the case from Session Two to Session Three, 
except that Group 1 (SM - min.) and Group 4- (SM - str . , PM - min.) show 
a decrease. The dropoff for Group 4 is particularly dramatic. In 
contradistinction to the ratio analysis, the data here would suggest 
that if significant differences were going to occur, they would show 
up most clearly in an (S^ ■ Sj^) rather than an (S 2 ■ S^) analysis. 

B. MODELING EFFECTS (S - S-^) and (S - S ) ANALYSES OF COVARIANCE. 

Table 10 presents a summary of the analyses of covariance for 

between group differences in all sessions with Total Probes as the 

dependent variable. While the F ratio for differences showed a trend 

in the predicted direction by Session Two (p ^ *25), it will be seen 

that a second training or treatment session was necessary before the 
differential effectiveness of certain treatments became sufficiently 
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Proportion of Probes toTotol Pupil Responses 




Teaching session 



Figure 1, Experiment III 

Adjusted Mean Frequency of Proportion of Probes 
to Total Pupil Responses Across Teaching Sessions 
for Each Experimental Group. 














SUMMARY OF THE ANALYSES OF VARIANCE FOR ADJUSTED MEAN DIFFERENCES BETWEEN 
ALL GROUPS OVER ALL SESSIONS WITH TOTAL PROBES AS THE DEPENDENT VARIABLE 





Level of Significance: **: p<.05 



powerful to be reflected at an acceptable level of significance (F - 
2.571; pC05). 

These modeling effects are illustrated in Figure 2. The pretest 
performance of each group on Probing was used in the adjustment for 
initial differences. The plot-points for each of the groups in 
Session Two are derived from the adjusted means differences between 
Session Two and Session One frequencies of Probes ; those for Session 
Three on the (S3 - S^) adjusted mean differences. Since each group 
started from a pretest score which obviously could not differ from itself, 
the curve for each group begins at zero, and thus the means entered in the 
figure did not have to be brought to a common point by the application of 
a constant. Needless to say, this is a real advantage in trying to pre- 
sent a meaningful picture of the results. 

Note that from Session One to Session Two, the groups perform as ex- 
pected. However, by Session Three, Group 4 (SM - str., Pm - min.) and 
Group 3 (MS - min.) have changed their relative positions on the predicted 
treatment continuum. Both groups it will be remembered, were exposed to 
perceptual models. They differed from each other in that Group 4 received 
discrimination training from E when viewing the self, whereas Group 3 
viewed both model and self -playbacks alone. 

Comparable analyses with Clarification as the dependent variable 
were also performed, and yielded results that correspond closely with those 
for the Probing analysis. A summary of the results of the covariance 
analyses carried out on Clarification appear in Table 11, and are illus- 
trated in Figure 3. 



- 114 - 



Adjusted mean difference 



Modeling Effects*. Probes 




Adjusted Mean Differences Between Teaching Sessions 
2 and 1, and Sessions 3 and 1, for Each Experimental Group. 



SUMMARY OF THE ANALYSES OF VARIANCE FOR ADJUSTED MEAN DIFFERENCES BETWEEN 
ALL GROUPS OVER ALL SESSIONS WITH CLARIFY AS THE DEPENDENT VARIABLE 
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O 




Level of Significance: ***: p^.025 



Adjusted mean 



Modeling Effects'. Clarify 




Teaching session 



Figure 3, Experiment III 

Adjusted Mean Differences Between Sessions 2 and 1, 
and Sessions 3 and 1, for Each Experimental Group. 



The most discernable differences between the two sets of analyses 
involve Groups 2 (Sm - max.) and 4 (SM - str., PM - min.) both of 
whom show decreases in response strengths on Clarification which are not 
reflected in Probing responses. Table L-1^ presents data which appear to 
resolve this issue. It will be noted that both of these groups show pro- 
portionate increases in the sub-categories of Probin g which counter-balance 
shifts in Clarification . For Group 4, there is almost a 10 percent gain 
in the lesser sub-categories from Session One to Two while Clarification 
in relation to Total Probes drops correspondingly. This is enough to 
change the rank order of this group. This shift, however, is not a signif- 
icant one. 

Similarly, Group 2 subjects demonstrated increases in Probing sub- 
categories from Session Two to Three, such that they show a fall off in 
Clarification, but retain a slight increase in Total Probes. 

Before specific tests on the hypotheses are reported, other general 
treatment differences must be discussed. Table 12 presents a summary of 
F ratios and significance levels for analyses conducted on all of the 
major variables involved in the study. It is evident that the only sig- 
nificant increases in response strength by Session Two were those for 
Pupil Answers (p^.05) . These are closely related to Total Pupil Respons e 
(rpA.TPR = *89), and together with a trend (p^.25) in Pupil Commen t were 
nearly enough to produce significant gains in the broader category (p 
for Total Pupil Responses^ . 10) . 

A summary of the analyses for Pupil A nswers appears in Table 13. 

Figure 4. which plots these adjusted m ean differences, is very similar to 
^See Appendix L. 
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TABLE 13 

SUMMARY OF THE ANALYSES OF VARIANCE FOR ADJUSTED MEAN DIFFERENCES BETWEEN 
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Level of Significance: 



that for Probes (Figure 2). This is not surprising in view of the strength 

of the association betv/een these two variables (rp^p^ = .91). 

An inspection of the F ratios for Total Probes and its sub-categories 
along with Non-Probes in the (S 2 - Si) analysis implies considerable 
response variability in the second session. The rank-order of adjusted 
mean differences for Total Probes (p(.25) is exactly as predicted. How- 
every, for those sub-categories which approach significance or show trends 
in the right direction, the order of treatments is somewhat variable, and 
it will be noted that there is also a discernable increase in Non-Probes 
(p<.25). By Session 3, however, rank-orderings appear to have been 

stabilized. 

The response category termed Intern Re peats Pupil Response (Repeats) 
surprisingly approaches significance in the (S 2 - Si) analysis (p^.lO) 
and in the (S 3 - Si) analysis, exceeds the .05 level of significance. A 
summary of the analyses of covariance and the relationship of adjusted 
mean differences for each group with all others appears in Table 14. 

These means behave in a manner analagous to Probing variables. 

This was surprising because the experimenters' sets about this var- 
iable were that it would occur fairly frequently during the pre-session 
tests, and if not extinguished or suppressed during training, would tend 
to "crowd-out" Probing responses. They were quite wrong on both counts. 
Intern Repeats were found to be more highly correlated with Probi ng (.65) 
than with Non-Probes (.35), and response strength increased significantly 
from Session One to Session Three, rather than decreasing. 

For the Experiment Two data mentioned earlier, in which subjects 
received treatments like those for Group Two in this study but taught 
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(3) Level of Significance: **: p<.05 

*; p<.10 

a: p<.25 



regular lessons in their classroonij Intern Repea ts correlated .54 with 
Total Probes , and .26 with Non-Probes . In addition, while initial re- 
sponse strength on Repeats appeared somewhat higher to begin with, there 
were no significant changes during training. For Experiment III on the 
other hand, all of those groups who were exposed to perceptual models 
during treatment showed significant gains both in the (S2 - S-^) and (S3 - Si) 
contrasts for training differences with the T statistic (see Table 8) . 

The Symbolic Modeling Groups, like those in Experiment Two showed no such 

gains , 

There seems to be little doubt then that these differences are due 
to modeling treatments, and are not explainable by other variables. Cer- 
tainly experimenter influence is not operating in favor of Groups 3, 4, 

5, and 6. If anything it could be expected to attenuate the results. 

An inspection of the model protocals in Table 5 shows that Repeats oc- 
curred in considerable strength. The possibility is that Intern Repeats 
became an Sjj which served to trigger Probing responses, i.e., repeating 
the pupil response gave the intern or model a brief time period during 
which he could relate a given probe to the subject-matter at hand. 

More simply, and perhaps in combination with the above, through 
contiguous association with Probing as exemplified in the models' perfor- 
mance, Repeats b ecame conditioned to certain pupil responses. If such were 
the case, then through modeling procedures, stimuli which had formerly been 
bound up with other verbal habit patterns came to act as cues to signal 
other behaviors such as Probing . 

C . FEEDBACK EFFECTS 

Overall treatment differences for feedback effects were analyzed 
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by considering (S 3 - S 2 ) mean differences with two covariates. As in 
the analyses for modeling effects, each of the major response categories 
were considered, but the focus now shifted to an examination of predicted 
differences between Self-Feedback (Group 3), Conf irmat ion (Group 4), 
Prompting (Group 5), and Combined Prompting and Confirmation (Group 6 ) 
procedures . 

An inspection of Table 12 data shows that significant differences 
were achieved only on the Clarification analyses. While Intern Repeats 
showed a trend (p<. 25), all ether analyses yielded non-significant F ratios. 

The summary of the analyses of covariance for feedback effects 
(S 3 - S 2 ) on Clarification is presented in Table 11. Figure 4 presents 
adjusted mean differences for the analysis on this variable and includes 
adjusted mean differences (S 3 - S 2 ) on Total Probes as well. The latter 
means are included to provide an illustrative comparison between the two 
response categories when modeling effects are controlled for by covariance 
adjustements . It will be seen that predicted order effects for these lat- 
ter analyses are wide of the mark in that Group 3 (Self -Feedback ) demon- 
strated greater gains than either Prompting or Confirmation Feedback 
Groups. This is somewhat surprising in view of results obtained in 
Experiment I in which teacher reinforcement of pupil participation was 
the dependent variable , and the Self-Feedback group showed decrements 
in response strength for Sessions Three and Four. The response pattern 
here is precisely the opposite. 

D. INTERVENING VARIABLES ANALYSIS 

As was pointed out earlier, T statistic comparisons among means for 
potential experimenter, age , and GRE performance differences were tested 
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and proved to be non-significant. A more sophisticated analysis in which 
analyses of covariance on Session Three means for selected major response 
categories (Session One mean scores as covariates) were also run. In 
these analyses for general linear hypotheses on all possible differences 
between say. Experimenter Irand Experimenter 2, on a given dependent 
variable such as Probing or Non-Probing . F ratios for row, column and 
interaction (row x column) effects are reported. This allows one to de- 
termine not only whether there were experimenter (row) effects on a given 
response category for any one of the six experimental groups, but also 
to obtain a second covariance analysis on F ratios with different ^ on 
main effects, i.e., even though row F ratios may be non-significant, thus 
telling us that there were no differential experimenter effects, column 

N 

F ratios will reflect the significance of difference between means on say. 
Total Probes across the six groups. These ratios will tend to be smaller, 
since the are usually considerably less. For example, in the analysis 
for potential experimenter effects, adjusted means for Groups 2, 4, 5, 
and 6 (those groups who were exposed to E at some point in treatment) 
were analyzed where Total Probes . Non-Probes and Intern Reinforces the 
Pupil adjusted means were entered into the analysis. The for row or 
experimenter effects were thus 1 and 62, those for column (treatment) 
effects 3 and 62, while those for row x column effects where one might 
find that a given experimenter was differentially effective with say. 

Group Six subjects, but not Group Two subjects, were also 3 and 62. Here- 
then, in column effects is a supplementary analysis for treatment dif- 
ferences with a reduced N and fewer degrees of freedom. 

Before these results are summarized, the rationale for selecting 
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Figure 4, Experiment III 

Adjusted Mean Differences Between Sessions 3 and 2 
for Probe and Clarification Responses for Each 
Experimental Group. 




Reinforcement . Probes , and Non-Probes for the analysis of intervening 



variable effects will be briefly outlined. 

Probes and Non-Probes were obvious choices because they constitute 
general measures of the dependent variable. An examination of Table 12 
will also show that Probes produced significant differences in the (S^ ~ 



analysis. Thus they seemed a reasonable choice. Non-Probes , of course, 
are the Ipgical opposites of Probes , and for this reason were also in- 
cluded. Reinforces was selected as the third variable because Ei had had 
considerable experimental experience in this area, and if differences 
apart from those on the major response categories were likely. Reinforces 
might well reflect them. In addition, reinforcement procedures are known 
to be responsive to sex differences, and since they aheapredominantly 
verbal behaviors, it was thought that they should also pick up GRE verbal 
score differences if any existed. 

Of all of the analyses for potentially differential effects: by 
experimenter, sex, perceptual model, GRE verbal and GRE non-verbal 
scores, only one significant result was obtained. Subjects in each 
group were dichotomized at the median into a high or low scoring group 
in terms of their GRE Verbal scores. The analyses of covariance on the 
three selected dependent variables showed that by the third session, 
low scorers did not Probe significantly more frequently than did the high 
scoring group (Fj^ 5 ^) = 5.239; p^.05) . The low scorers were not, 
however, localized in any particular experimental group, for interaction 
analysis yielded a non-significant F ratio. The inference that the low 
scoring group may also have Probed significantly less frequently than 



analysis, and yielded trend 




other two types of 
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the high scorers Is Incorrect, since the row F ratio on Probe s was non- 
significant. Column effects for Probing were, however, significant 
(jj j^=2.699; p^OS), providing further support for the(S 3 - 83 ) anal- 

ysls on Probing reported earlier. 

There were no significant sex differences. However, here again, 

column effects for Probing were significant (F5,95 “ 2.735j p<.05). 
Similarly, analyses for the differential effectiveness of one or more 
models yielded nonsignificant differences. Column effects for Probi ng 

were also non-significant (F3,59 “ 1.549J p<25). 

Finally, GRE quantitative scores and experimenter effects proved to 

be non-significant, although added support for significant differences 
between the experimental groups on Probing was again found (F 3 , 52 = 2 -834; 
p<.05) on the experimenter-effects analysis. 

Having presented evidence which indicates that the significant 
differences between groups on the covariance analyses for major response 
categories of dependent variables are not due to non-treatment effects 
such as differential experimenter performance, intelligence or other 
variables of a similar nature, this discussion now turns to specific 

tests of the hypotheses. 

E. SPECIFIC TESTS OF THE HYPOTHESES 

The selection of the most appropriate test for specific contrasts 

following significant F ratios is problematic, and requires brief dls- 
CUSS ion. 

There is general consensus among statisticians that the most 
appropriate test for a priori hypothese following significant F ratios 
is the t test (c.f. McNemar, 1962; Winer, 1962; Ferguson, 1959). However 



it is also known that when t is used for multiple comparisons among means, 
the probability of committing a Type-1 error is approximately .05. This 
suggests that, in these cases a more rigorous basis than usual should be 
required for the rejection of the null hypothesis. One could avoid the 
t test altogether and use the ^key Test or the Scheff^ method. But as 
Winer (1962, p. 89) points out, these a posteriori tests may lead to the 
commission of Type-2 errors* even when one is "data- snooping." An 
additional complication involving the T test is that it requires equal 
N's and this would mean an appreciable reduction in sample size for the 
Experiment III data as N varies from 17 to 20, i.e., 34 cases would have 

had to be discarded in such an analysis. 

Following Ferguson (1959, p. 238) the rejection region for the t 
statistic'-' was examined in .terms of the more rigorous 10/K (K-1) percent 
level, where K is equal to the number of groups. In terms of Experi- 
ment III data, the adjustment was such that in order for the t to be re- 
ported as being significant at the .05 level, it would actually have had 
to reach the .017 level in conventional tables for critical values of Jt. 

Rather than working out the adjustment for all other appropriate sig- 
nifiCancec levels, it was decided to double the required level of signi- 
ficance for a given t value. Thus in Table 15, where the significance 
level for t is reported to be at the .05 level, it is actually at the 
.01 level in terms of conventional tables for t. Since the appropriate 
_t test for a priori hypotheses is for one- tail of the distribution only, 
the 10/K (K-1) adjustment has been made by reading off all relevant 
values for two- tail tests, and reporting them as being one-tailed tests. 
This procedure may thus be seen to be slightly biased in favor of rejecting 
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TABLE 15 

BETWEEN-GROUP (TREATMENT) COMPARISONS AND CONTRASTS 
FOR DIFFERENCES BETWEEN ADJUSTED MEANS FOLLOWING 
SIGNIFICANT F RATIOS BY ANALYSES OF COVARIANCE 



Response Category and 
Type of Analysis 


Direction of 
Differences 


Student’s t 
for df of 
100 


Adjusted 
Significance 
Level ( S') 


Probing (S^ - S, ) ; 


Group 6 


Group 1 


3.158 


.005 




Group 6 


Group 2 


2.U95 


.01 


F = 2.57 


Group 6 


Group U 


2 . 1 U 8 


.025 


5, 101 


Group 5 


Group 1 


2.159 


.025 




Group 5 


Group 2 


1.501 


.10 




Group 3 


Group 1 


2olll 


.025 




Group 3 


Group 2 


i.Uoo 


.10 


Clarification 


Group 6 


Group 1 


2.881 


.005 




Group 6 


Group 2 


2.816 


.005 


( ) I 


Group 6 


Group U 


2.319 


.025 


F = 2.962 


Group 5 


Group 1 


1.938 


.05 


5, 101 


Group 5 


Group 2 


1.853 


.05 




Group 3 


Group 2 


2.319 


.025 



Intern Rereats Pupil 
Response (S^ - S^); 

F 5,101 = 2 .U 92 


Group 6 
Group 6 
Group 6 

Group 5 
Group 3 


Group 2 
Group 1 
Group 5 

Group 2 
Group 2 


3.221 

2.678 

2.013 

1.337 

1.900 


.005 

.005 

.025 

.10 

.05 


Pupil Answers 


Group 6 


Group 1 


3 . 13 U 


.005 


(S« - Sj; 


Group 6 


Group 2 


2.178 


.025 


2 1 


Group 5 


Group 1 


2.378 


,025 


F 5,100 = 2 . 3 UI 


Group 5 


Group 1 


1.378 


.10 


Clarification 


Group 6 


Group 2 


3.015 


.005 


(S.. - S«): 


Group 6 


Group 1 


2.682 


.005 


\ 3 2’ 


Group 6 


Group U 


2 .U 9 U 


.01 




Group 3 


Group 2 


1.180 


.15 



^®'^The adjusted probability levels reported above are based on a 
more rigorous r-Jection level (lO/k (k-l), where K is the number of 
groups) than that for conventional p values (c.f. Ferguson, 1959* PP 237- 
238 ). The adjusted p values are for one-tailed tests. 
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a given hypothesis when in fact it should be accepted. 

It does> however > avoid the extremes of the T test on one hand and 

the unadjusted t test on the other. 

Hypothesis 1 (a) ; states that, "Perceptual Modeling procedures will 
produce significantly greater changes in the response strength of de- 
sired behaviors (Probing responses) than will Symbolic Modeling proce- 
dures." The statistical tests of this hypothesis are therefore t tests 
in which Group Three (Minimal Perceptual Modeling) is pitted against 
Group One (Minimal Symbolic Modeling) , and Group Five (Maximal Percep- 
tual Modeling) , is compared with Group Two (Maximal Symbolic Modeling) . 

Table 15 which reports the results for these and all other contrasts 
show that t test results for (S 3 - S^) mean differences among the rele- 
vant groups on Total Probes , generally support the hypothesis. 

Group Three (PM min.) subjects, as predicted showed significantly 
greater gains than did Group One (SM min.) subjects (^ for df of 100 
2.111; p-^.025, one-tailed). Group Three subjects also tended to be 
significantly different from the Maximal Symbolic Modeling subjects 
(Group Two) on Total Probes (t = 1.40; p <.10; one-tailed). Maximal 
Perceptual Modeling procedures (Group Five) led to significantly great- 
er performance gains on Total Probes than did the Minimal Symbolic 
Modeling treatment (p < .025) , and tended to be superior to Maximal 
Symbolic Modeling (Group Two) as well (p < .10). 

The contrasts on adjusted means for Clarification as the dependent 
variable led to similar results, though at greater significance levels. 
Again, Group Three showed significantly greater gains in the desired 
behavior than did Group One (p ^ *01) > ^nd was also significantly 
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superior to Group Two (p ^ .025) . Group Five (PM min.) was again sig* 
nificantly different from Group One (SM min.) (p ^.05) and unlike the 
analysis for Probes where it fell just short of significance, in Clari- 
fication . Group Five proved to be significantly different from Group Two 
(p < .05). 

Added support for the above findings may be seen in the results 
for Intern Repeats . As was explained earlier, differences were not 
predicted for this variable, but are due to treatment effects, and move 
in the same direction (Group Five - Group Two, p ^.10; Group Three - 
Group Two, p <.05). Finally, as Figure 1 shows, when ceiling limits 
on Probing are taken into account, the differences are consistently 
in favor of the hypothesis even though probability levels may not be 
safely stated as the means are unadjusted. 

The results then are quite consistent. With the effects of prior 
experience partialled out, it was found that by the Third Session, Mini- 
mal and Maximal Perceptual Modeling treatments were generally more 
effective than equivalent Symbolic Modeling procedures in increasing 
Clarification responses in particular and Probing in general. 

The results are attenuated somewhat by the lack of significance 
between the Maximal Symbolic and Perceptual Modeling conditions on 
Total Probes . It is concluded then that the first hypothesis is par- 
tially supported by the data. 

Hypothesis 1 (b) : states tha t a combination of Perceptual and 
Symbolic Modeling procedures will prove to be more effective than either 

Perceptual or Symbolic procedures . alone. 

The test for this hypothesis required that comparisons be made 
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between Group Six (SM - Str., PM - Max.) and Groups Five, Three, Two 
and One. Groups Five and Three represent Maximal and Minimal Percep- 
tual Modeling respectively, and Groups Two and One are their counter- 
parts in Symbolic Modeling. 

Table 15 summarizes the t test results for these comparisons. It 
was found that on the (S 3 - S^) analysis. Group Six consistently and 
significantly showed greater gains on Total Projie^, Clarificatio n, and 
Intern Repeats than did either of the Symbolic Modeling conditions. 

These differences as shown, exceed the .01 probability level. 

While Group Six was also significantly different from Group Five 
on Intern Repeats (p <.025), the former did not in general differ from 

either of the Perceptual Modeling Groups in question. 

It is concluded that the second hypothesis (lb) is partially sup- 
ported by the data. Combined Perceptual a nd Symbolic Modeling procedures 
proved to be significantly more effective than did Minimal or Maxima l 
Symbolic Modeling treatments alone. There is no consistent evidence 
to indicate that the Combined procedure is superior to either 

Minimal or Maximal Perceptual Modeling alone. 

H ypotheses 2 (a’^ and 2 (b ) : relate to feedback procedures, and 

state that nombined Promoting and Confirmation Feedback (Group Six) will 
yield greater gains in the desired behavior than either Prompting Fe ^- 
back (Group Five) , Confirmat ion Feedback (Group Four) , or Self-Feedback 

(Group Three) • 

The only analysis relevant to these considerations is the (S 3 - S 2 ) 
analysis in which feedback effects are considered independently of the 
main analysis for modeling effects by covariance adjustments for prior 



effects (Sp and initial modeling effects (S 2 ). Of the relevant analyses 
of covariance that were carried out in this manner, only that on Clari- 
fication proved to be significant. The t test for comparisons among 
means supported only one of the sub-hypotheses: Combined Prompti ng 

and Confirmation Feedback (Group Six) proved to be significantly differ- 
ent (p(.Ol) from the Confirmation procedure (Group Four). 

Group Three ^Self-Feedback ) subjects performed well beyond expecta- 
tion. In fact, the adjusted means for the groups on this analysis show 

that predictions were generally wide of the mark. 

The hypothesfflon feedback effects are there foB not supported by 

the data. 

Hypothesis 3 (a) is a general hypothesis with particular relevance 
for training experiments where the experimenter wishes to identify the 
optimally effective treatment condition. It states that the optim^ 
combination of presentation and feedback procedures will be one in 
which Combined Symbolic and Perceptual Modeling procedures in the pre- 
sentation phase are combined with the Prompting and Confirmation Fe^- 
back condition (Group Six). 

A complete evaluation of the hypothesis requires an examination of 
(S 3 - Sj^) differences on all relevant variables entering into Probing. 
Table 15 shows that Group Six subjects consistently and significantly 
realized greater gains on Total Probe_s . Clarification, and Intern 
peats , than did Groups One, Two or Four. Group Six also showed signi- 
ficant gains over Group Five on Intern Repeats . These differences ex- 
tend from the .005 to .025 level. 
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The differences between Group Six and Groups Three and Five are not 
significant on the main variables. However, as Figures 1 through 4 amply 
demonstrate. Group Six subjects uniformly showed the greatest gains, and 

achieved the highest mean scores. 

Hypothesis 3 (b) is a general hypothesis relating to performance 
curves, and states that performance gains from Session One to Session 
Two will be significantly greater than those from Session Two to Session 
Three, regardless of experimental group. 

To test this hypothesis, adjusted meani difference scores for all 
groups from Session One to Session Two on Probing were averaged and then 
compared with the overall mean difference of the groups for Session Two 
to Session Three. The t test for correlated means was a highly signi- 
ficant 21.32 (p <.005, two-tailed). An inspection of the data indi- 
cated that a similar analysis of scores on the Clarification variable 
would have yielded even larger differences, so no further tests were 
carried out. 

An examination of the figures already presented will show that as 
predicted, the gains from Session One to Session Two on all of the 
major variables (excluding Non-Probes) are demonstrably greater than 
those from Session Two to Session Three. It is therefore concluded that 
the hypothesis is supported by the data. 
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DISCUSSION OF RESULTS AND IMPLICATIONS 
FOR FURTHER RESEARCH 



The general assumption from which hypotheses were derived was that 
the rate and level of learning a given teaching strategy varies as a 
function of the mode of model presentation. Specific predictions were 
based on theoretical considerations which suggest that the differential 
effectiveness of varying model and associated feedback procedures stems 

from their distinctive cueing properties. 

The effects of Symbolic and Perceptual modeling together with 
certain feedback conditions were explored in terms of their ability to 
increase the distinctiveness of relevant cues. Training procedures 
were varied along a continuum of increasingly available cues and in- 
formation on the criterion behavior. 

Beginning with a common base of written instructions that described 
the desired behavior, the minimally effective dondition consisted of 
repeated exposures to the written instructions, and pictorial repre- 
sentations of subjects' prior performance. The power of successive 
treatments was increased by the addition of a verbal discrimination 
training on relevant cues and the inclusion of perceptual models who 

portrayed the desired responses. 

The treatment continuum extended then from a condition in which 
the criterion behavior was described through written instructions 
only (Minimal Symbolic Modeling), to the addition of verbal instruc- 
tions on relevant cues XMaximal Symbolic Modeling) , and then to 



Following set induction, they simply viewed playbacks of their prior 
performance, then the model tape, alone. 

In establishing a treatment continuum on theoretical grounds, these 
two groups fell on either side of the symbolic -perceptual demarcation 
line. The differences between them were significantly greater than 
for any other immediately adjacent pair of groups. Group Three S's 
achieved significantly greater gains than did Group Two S's on Clari- 
fication (p < .025), Intern Repeats (p < .05) and tended to maintain 
these differences on Total Probes (p ^ .10) in the (S^ - S^) analysis. 
These differences support the distinctions made between the two types 
of modeling on the basis of their cueing properties. 

A notable difference between the Maximim Symbolic (Group Two) and 
Perceptual (Group Five) modeling conditions in terms of the ease of 
training should be mentioned. Considerably less finesse and effort 
jijqtcq required of E when he provided model— based discrimination train- 
ing than when his comments were based on subjects' tapes. In the 
Perceptual Modeling conditions, a high frequency of Probing behaviors 
occurred and were modeled in a clear-cut manner. This increased the 
distinctiveness of relevant cues and discrimination training under 
these conditions appears to have been more efficient. 

Finally, in terms of treatment differences between specific groups, 
it was concluded that Group Six, which represents the most powerful 
combination of Symbolic and Perceptual Modeling procedures, constitutes 
the optimal training procedure. The evidence in support of this con- 
clusion is drawn from the consistent superiority of this group across 
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all types of analysis. 

Certain qualifications do, however, attach to specific hypotheses, 
and these must now be considered. Experimentally, the fact that all six 
treatment conditions produced significant changes in the response strength 
of the dependent variable could well have turned out to be a mixed bless- 
ing. The major consideration in research of this kind is the differen- 
tial effectiveness of various treatment conditions. If the least pow- 
erful procedure produces changes which bring subjects close to the asymp- 
tote of performance on a series of behaviors with a ceiling limit, then 
the experimental utility of the dependent variable is severely curtailed, 
as potentially significant differences among treatments are wiped out. 
While this did not happen in this experiment, such considerations are 
relevant, for on the treatment continuum from Minimal Symbolic to Com- 
bined Maximal Symbolic and Perceptual Modeling (Group One through Group 
Six), those groups immediately adjacent to each other and within each 
set of modeling treatments did not as a general rule differ significant- 
ly. 

The empirical nature of the dependent variable appears to have been 
an important factor here. What appears to have happened is that the 
greatest gains in Probing techniques occurred in Clarification, which was 
at the same time, the most frequently occurring Probing behavior prior 
to training. The evidence indicates that in the thick of classroom in- 
teraction, the Probing techniques which are most likely to be used fre- 
quently, are those which are already prepotent in teacher's response re- 
pertoire. Unless training is directed specifically towards shifts in 
this response hierarchy, increases in Probing regardless of their scope 
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will follow the original distribution of responses. Had the models re- 
ceived discrimination training more directly focused on each of the 
sub-categories of Probing , rather than upon Probing in general, it is 
likely that treatment differences would have been correspondingly 
sharper . 

These outcomes underscore the experimental utility of Bandura's 
(1963 a, c.) designs, where models are trained to emit highly unusual behav- 
iors. The contingencies in training research are, however, quite dif- 
ferent, and it may frequently prove to be impractical to seek out unique 
operants to serve as dependent variables. Viable teaching skills and 
strategies can be expected to have a certain "survival value", having 
been tested out over the years, and in developing and employing them 
as dependent variables, one can expect them to be occurring in some 
strength prior to treatment. At the very least, their nature is such 
as to make them relatively "easy" to learn, since the central verbal 
elements of these skills will already be a part of the response-reper- 
toires of adult subjects. The operant strengths of likely skills and 
strategies should be assessed with naive populations before experiment- 
ation begins. 

Finally, the above re-examination of Probing as a basic question- 
ing technique clearly implies that pre-lesson planning must be highly 
systematic and concrete if the Probing which occurs during interaction 
is going to go beyond the obvious. Probing is not a simple skill that 
can be easily acquired. It might be pointed out in this regard that 
even though approximately one-half of their responses could be classified 
in the Clarification category, learning curves based on group means 
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by session did not approach asymptote by the second session as they 
apparently did in Experiment I where teacher reinforcement for selected 
pupil responses was the dependent variable. Indeed, indications are 
that in the current experiment, further trials would have led to further 
gains, as the learning curves for each group on Total Probes (Figure 
2) were still on the rise in the Third Session. 

General Implications for Further Research ; This study represents 
an initial attempt to adapt modeling procedures to teacher training. 

It is obvious that the last word is yet to be said on this topic. 

The question of the transferability of the skills learned under 
the training conditions outlined here is a central issue, and is yet to 
be systematically explored. There is, for example, evidence to indi- 
cate that the more closely one is able to approximate actual perform- 
ance conditions in training, the more effectively will such procedures 
contribute to later performance (Roshal, 1961). This suggests that in 
replications of this kind of study, experimental groups be included in 
which interns or teachers receive modeling experiences while teaching 
regular classes, and that models be presented teaching in comparable 
conditions. 

Variables such as the amount of practice, the optimal number of 
cues emitted by models, sex differences in models and the sequencing of 
steps in treatment were controlled in this study rather than system- 
atically varied, and appear to be worthy of future investigation. In 
addition, the provision of multiple models in training constitutes an 
inviting research possibility. Interns in the present experiment stated 
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that they found the second exposure to the same model somewhat repeti- 
tious. One could hypothesize that either: (a) the provision of two or 

more different models demonstrating the same techniques under standard 
conditions would enhance learning because they were better able to 
maintain attention to salient cues, and because they provided greater 
variety in the initial presentation of tasks; or, (b) one could hyp- 
othesize that the provision of multiple models would impede the rate 
of learning because of their initial stimulus -novelty value (leading 
the subject to cue-in on irrelevant details). Such associations must 
be allowed to extinguish before discrimination on the relevant cues can 
begin to increase in strength. 

Finally, the nature of the dependent variable is such that it would 
probably be worthwhile to assess its relevance in terms of performance 
criteria. It seems to have considerable face validity. The question 
remains however, does the use of such techniques produce demonstrable 
changes in pupil performance? 

Generalizabtlitv of the Data : There are at least two general ques- 

tions one might ask in attempting to generalize the results of this 
study.. First, what is the probability that the results obtained here 
would hold for other teacher samples under analagous. experimental (or 
training) conditions. 

Excluding the foreign-language teachers, the sample for this study 
constituted the entire Stanford Intern class. Their performance on 
the Graduate Record Examination marks them as a relatively select group. 
Their responses to the questionnaire, and frequent informal communica 
tions to E and other personnel connected with their training but not 



with the experiment, suggest that they were well-motivated, and per- 
ceived the training as highly useful. Most inq>ortantly perhaps, all of 
the subjects were beginning teachers, and had not been long in the pro- 
gram. They had not as yet been given even limited responsibility for 
a classroom of students. 

It is possible that individual differences such as those between 
neophytes and experienced teachers could act in such a way as to produce 
differences among the Perceptual Modeling treatments. For example. Mini- 
mal Perceptual Modeling might prove to be less effective with experienced 
teachers than with neophytes because the former may have developed re- 
sponse patterns which are highly resistant to extinction. In such cases, 
treatment conditions such as those for Group Four where Minimal Percept- 
ual Modeling was supplemented with discrimination training (or retrain- 
ing) based on subject playbacks might prove to be more effective. 

It is not, however, the intent of this closing discussion to gen- 
erate new rationales. The above illustrations are simply included to 
show that first, the results of the present study must be viewed within 
the limitations set by the design and secondly, the study may serve a 
heuristic function in generating new areas for research. 

The second issue that is relevant to considerations of generaliza- 
bility, relates rather more directly to the kinds of skills or stra- 
tegies one might wish to transmit to teachers. While Perceptual Model- 
ing techniques proved to be superior to written or verbal instructions, 
where basic questioning techniques were Involved, it is doubtful that 
this superiority would be maintained across all teacher behaviors. It is 
not unlikely that certain behaviors can most efficiently be transmitted 
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simply through verbal or written instructions. 

On the other hand, there is some evidence to indicate that in cer- 
tain situations, real-life models may be more effective than filmed 
models. While Bandura, Ross and Ross (1963b) found that filmed and real- 
life models led to comparable changes in aggressive behavior, recent 
work by Bandura and his associates (1965) indicates that in cases where 
emotional responses of the subject are central to behavior change (e.g., 
fear of dogs; phobic behavior) real-life models produce greater response 
shifts than do filmed models.^ Filmed portrayals would appear to be 
particularly effective where non-verbal behaviors involving gestural 
responses are relevant to the acquisition of a skill, or where the S® 
that triggers the desired behavior is clearly discriminable as was the 
case in> this study, and where the behavior is sufficiently complex - 
(say, conducting a chorus) that precise written and verbal instructions 

are inadequate. 



^It should be pointed out here that in distinguishing between 
Real-Life and Symbolic Models, Bandura and Walters (1963b, p. 49) con- 
sider filmed demonstrations of a given behavior or strategy to be 
exemplars of the latter. 

By varying the distinctiveness of cues from written instruc- 
tions to filmed demonstrations, and not considering Real-Life models 
in the subsequent classification of treatments, this distinction was 
blurred in the current study. 

All of the treatments in this study constitute forms of Symbolic 
Modeling, so that the two central forms considered might well have been 
labeled Symbolic -Symbolic and Symbolic -Per ceptual . 

This latter procedure corresponds more closely with the current 
literature on modeling as a whole. 
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Conclusions: Finally ^ it is concluded that the results of this study 

support the assumption that the rate and level of learning a given 
teaching strategy vary as a function of the mode of model presenta- 
tion. More specifically, there is evidence to indicate that Percep- 
tual Modeling procedures are characterized by distinctive cuing 
properties which tend to recommend them over Symbolic Modeling pro- 
cedures for use in training contexts analagous to those described in 
the experiment. 

Due to the lack of clear-cut support for all of the experi- 
mental hypotheses, these conclusions are tempered with the qualifica- 
tion that while treatment differences occurred as predicted, they were 
in the main concentrated at the extremes between the two types of model 
ing. 
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The results of these experiments may be summarized in terms of(l) 
their contributions to the analysis of teaching behavior and its effects; 
(2) the information they yield about the effects of training variables 
on producing certain categories of teaching behavior; (3) the problems 
they suggest. Only the second of these categories is the direct yield 
of the experimentation. The first and third categories represent ob- 
servations made during the course of experimentation and data analysis. 
Not all of the comments made in these two categories have been tested 
in the strict sense. They are offered here to alert our colleagues to 
methods and research questions which may interest them. 

The Significance of Experimenting on Teaching Behavior . 

Those experiments represent the few, if not the only , experiment 
on teaching behavior in which the conditions of a true experiment have 
been maintained. They represent a proof in principle that such exper- 



Thus, the possibility develops that we may now test theories of 
teaching in the same rigorous way that some other empirical theories 
have been tested. The vision of an empirical science of teaching 
promises to be realizable. The development of such a science through 
systematic experimenting will inevitably change the nature of teach- 
ing practice. Three moderately successful experiments is a small seed 
from which to grow that flower of hope and expectation. But the demon- 
stration that such experiments can be done successfully was the first 
necessary step which, until it had been taken, made visions of the 
hopes of a rigorous science of teaching. 



imentation can be conducted 
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The Technology of Experimenting on Teachi ng Behavior. 

The portable videotape recorder was the single most significant 
contributor in directing these experiments. This device provided a 
means of providing feedback on a trainee's behavior in a way that no 

other means provided. 

Human observation and comment has been largely ineffective for 
many reasons, principally that the observer is a limited data-gatherer, 
and the interaction between trainee and observer is fraught with prob- 
lems. Films and other types of video equipment are cumbersome and the 
time-lapse to viewing is greater (this latter characteristic may be 
less significant than we originally believed). 

The portable video recorder makes instant playback a reality, is 
unobtrusive after familiarizing teachers and their students with it, 
and is flexible in use, easily moved (an important characteristic when 
trainees are teaching in widely separated locales), durable, and rel- 
atively economical. 

None of these characteristics, however, would have had great 
psychological significance if the process of viewing oneself on video- 
tape did not act as an effective stimulus. Although we cannot prove 
it, it seems that such viewing is analogous to seeing oneself as one 
performs. The trainee seems to recreate the teaching experience, with 
sufficient distance, however, to objectify it to some degree. 

Also, the video recording is an objective record available both 
to the trainee and independent observers. Although a trainee may dis- 
tort for himself what the record may seem to portray to another, this 
kind of a record cannot be dismissed or distorted as easily as the 
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reports of observers on the actions of teachers. 

These two aspects of the video recording, its power to recreate 
the original experience and its objectivity, appear to provide the nec- 
essary conditions for using the video record as a training device. How 
the record may be used is an important problem for study. In the ex- 
periments reported here the video record was used as part of a feedback 

system and as a way of portraying the desired teaching behavior. 

Results of Experiments 

The results of the experiments described in detail in the preced- 
ing sections may be summarized under the following headings: (1) what 
were the most effective training variables} (2) what seemed to be the 
least effective variables} (3) what results appear to be inconsistent 
with other psychological research. 

An important disclaimer is necessary here. Significant training 
effects were obtained in all experiments; i.e., trainees improved on 
the criterion behavior across teaching sessions in all treatments. 

Even though there is an unaccounted for drop in some groups in the last 
session, all groups performed at a higher rate at the end of the exper- 
iment than at the beginning. 

There is no "pure" control group in any of these experiments, a 
group that is tested but receives no relevant training. Our control j 

I 

groups all received some type of training which was relevant to, but ^ 

j 

not necessarily specific to, the behavior to be learned. For example, 
in Experiment I, a group viewed its own performances on videotapes and j 

evaluated them on the dependent variable (reinforcing students' par- 
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ticipating behavior). This group's treatment differed from that of the 
other groups in that trainees' attention was not focussed on the de- 
pendent variable. Similarly, in Experiment III, all groups received 
some form of modeling instructions, no group receiving no such demon- 
stration. The "weakest" treatment consisted in a detailed description 
of the desired behavior. 

In each of these cases it is apparent that all trainees in these 
experiments were receiving some form of treatment relevant to the be- 
havior to be acquired. Since we do not at present know the specific 
factors which may produce the desired behavior changes, it is not un- 
likely that some aspects of a critical variable, or, perhaps, even the 
critical variable itself, may be included in these supposedly "weak" 
treatments. For example, viewing oneself on videotape may be the nec- 
essary condition for producing certain behavior changes. A person may 
need to see himself behaving to conceptualize how he is behaving and 
to contrast a desired performance with his observed performanca. 

Thus, in Experiment I, a trainee might have no idea of the manner 
in which and the frequency with which he reinforces students' partici- 
pating in class. Seeing himself on videotape may be a necessary cue 
to alert him to the need for and ways of modifying his behavior. When 
the behavior being learned is a common social behavior, such as reward- 
ing other people's verbal behavior, the simple viewing of one's own 
performances may be sufficient to initiate a behavior change. Although 
we obtained significant differences across treatments in Experiment I, 
the line of reasoning presented here suggests that these differences 
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might have been even larger had we used a "no-treatment" control group. 

On the other hand, when, as in Experiment III, the behavior to be 
learned may not require observing one's own performance, differences 
are even less likely to emerge. In that eixperiment, the behavior to be 
learned may be described by giving a rule: "Whenever a student offers 
an answer in class, ask him another question which requires him to go 
beyond the answer that he has just given." It seems plausible that a 
trainee could follow this rule rather easily without having observed 
his own performance to determine whether he is already following It. 

Observing his own performance may cue him to instances when he 
failed to observe the rule. Observing a model's performance may cue 
him to the variety of forms applying the rule may take. Thus, self- 
viewing and viewing a model's performance enhances learning, but the 
critical learning occurs once the subject has grasped the rule. 

The point of this line of argumentation is that the results of the 
presented research demonstrate that some variables are probably effect- 
ive, but the reader is to be aware that the relative effectiveness of 
these variables is probably underestimated in some cases, and in others, 
ma y not have been. The summary of the results described in the follow- 
ing section should be evaluated in the light of these qualifications. 
Most Effective Variables ; 

It seems clear that the single most effective variable was a form 
of self -viewing, accompanied by prompting by an experimenter during the 
self -viewing. The most powerful treatment in Experiment I was one in 
which the trainee viewed his own performance and received reinforcement 
and cue discrimination training from an experimenter. Similarly, the 
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most effective treatments in Experiment III involved the use of an ex- 
perimenter in a feedback condition. 

The most effective variable for describing a desired behavior ap- 
pears to be a modeUng condition in which the behavior is portrayed, 
and in which the subject views the model's performance while being 
cued by an experimenter on the significant aspects of the model's be- 
havior. The results of Experiment III support this statement. In 
that experiment, any condition in which the experimenter was present 
while the subject viewed a model tape was always more effective than 
those conditions in which the subject viewed the tape alone. The 
single most effective condition combined a modeling experience with 
a feedback condition, both of which had an experimenter present who 
cued the trainee on desired behavior. 

Although the experiment has not been performed which effectively 
separates out the effects of these two kinds of variables, it seems 
clear that for producing some kinds of behavior change, a modeling 
and feedback condition with an experimenter present during both phases 
is a powerful treatment. 

As pointed out previously, this result does not appear strongly 
enough in the experimental results to accept it without qualifications. 
Such a treatment might be demonstrated to be highly effective when the 
behavior being learned is not easily cued by simple instructions. It 
also seems likely that a feedback condition will probably be more ef- 
fective in producing those behaviors for which the trainee already has 
component responses in his repertoire. 
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Least Effective Variables; 



The results of Experiment II ^ although they are difficult to in- 
terpret and must not be interpreted as a conclusion (since no signifi- 
cant differences were obtained), are consistent with a fact that was 
apparent in the first two experiments. In Experiment II, time -lapses 
between the time of the occurrence of the original behavior and the 
time when the subject actually viewed his performance were uncorrelated 
with measured changes in behavior, even though these time lapses ex- 
tended over days and weeks. Again, repeating the warning not to inter- 
pret the acceptance of the null hypothesis as demonstrating an experi- 
mental effect, these results are so striking that they must raise a 
question about an important variable--the immediacy of feedback. 

In a teaching situation, the teacher receives a certain amount of 
feedback upon his performance from the behavior of students. This in- 
formation is received more or less immediately after the enactment of 
the teacher's behavior. For example, if the teacher asks a question 
and receives an appropriate answer, the teacher assumes that his pre- 
sentation to this point has been effective. Similarly, other types of 
student behavior, such as attention and interest, give the teacher in- 
formation on his teaching behavior. 

On the other hand, when supervisors evaluate a teacher's perfor- 
mance, such information is usually given at a point in time after the 
behavior has actually occurred. Ordinarily, the shortest time between 
the enactment of the behavior and feedback from a supervisor on that 
behavior is the time between when the behavior occurs and the end of a 
teaching period. In the chapter on theory for this report, we pointed 
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out the limitations of giving feedback verbally about behavior which 
occurred at an earlier point in time. However, the use of the video- 
tape recorder suffers from a similar limitation— it can be used only 
after the actual teaching itself has been completed. 

However, unlike the verbal report, it appears that this time-lapse 
is not a critical factor. In both Experiment I and II trainees re- 
ceived information on their teaching performance hours, and in the case 
of trainees in Experiment II, days and weeks after the actual perfor- 
mance. Experiment I indicates that this time lapse was not a critical 
factor. In that experiment the time lapse was equated across these 
treatments. In all treatments, feedback was delayed by several hours, 
yet striking changes occurred. It might be argued that the results 
would have been greater had the feedback been closer in time. The 
plausibility of this argument is weakened by the results of Experiment 
II. In Experiment II, where the time-lapse was systematically varied, 
but the training was held constant, no effective differences were ob- 
tained. When the results of these two experiments are put together in 
this fashion, it appears that the immediacy of the feedback, where 
immediacy is measured in terms of time, is not a critical factor. 

The explanation for this may be that the videotape playback re- 
instates the trainee's performance for him. The whole experience 
of viewing oneself on the videotape is quite different from receiving 
information from a second person about one's performance. The char- 
acter of the feedback experience has changed drastically. Whatever 
factors may be involved in this new experience are sufficiently dif- 
ferent so that the factor of immediacy is no longer relevant. 
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This observation, if confirmed in further experimentation, is both 
theoretically and practically important. Theoretically, it suggests 
that feedback processes depend essentially on the information process 
ing characteristics of the subject, not on the time-space relations 
of the feedback to the subject. Practically, reducing or eliminating 
the need to provide immediate feedback makes a wide range of feed- 
back systems possible, which are easier to manage and more economical. 
Consistency with Results of Other Psyc hological Experimentation; 

It is clear from the results of Experiment I, with the exception 
noted in the immediately preceding section, that the feedback variable 
is highly effective in changing behavior. As was pointed out earlier, 
reinforcing desired behavior produces behavior changes. The problem is 
to determine what kinds of reinforcements, in what amount, and dispensed 
at what rate, are most effective in changing which kinds of teaching 
behavior. Practically, a simple decision rule seems to be: Always 

include a feedback system in which the trainee v iews his own perfor- 
mance with sup ervision. 

It is important to note that conditions in which the feedback was 
provided without the assistance of an experimenter were always less 
effective than those in which an experimenter was present. It is dif- 
ficult in this study to separate out the reinforcing properties of the 
experimenter's behavior and its cue-discrimination characteristics. 
Thus, we have a problem which has faced psychologists for many years. 
Although we have preferred to think of the feedback system as essen- 
tially information-providing, it is apparent, from naturalistic ob- 
servations during the experiment, that this information has reinforcing 
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characteristics which are only moderately correlated and may be un- 
correlated with the information characteristics of the feedback. 

It may be that feedback is most effective when it occurs closest 
to the time the behavior is to be performed next. This possibility 
makes sense when we recall that a teaching behavior is performed at 
one time and may not be repeatable until the next day. This hypothesis 

is easily tested. 

The modeling treatments in Experiment III were not as powerful as 
we had hoped. We suspect that the critical factor in this case is the 
nature of the dependent variable. Probing behavior is a common, if not 
frequent, form of verbal interaction, particularly among well-educated 
people, like the trainees in this experiment. Furthermore, these 
trainees have been exposed to many models of this same behavior in their 
own very recent college experience, and perhaps may be perceiving models 
of it during their current training period. It is not unusual for un- 
iversity teachers to query students on their thinking. The "give-and- 
take" of many a seminar consists in exchanges of probing-type questions. 
Since our experiment is a "real-life" experience in the sense that the 
behavior to be learned is to be translated immediately into "real-life" 
situations -- (actual teaching), and since it is observed and used in 
everyday experiences, it may be that the effectiveness of modeling has 
simply been attenuated. Modeling may be more effective with behavior 
changes where the behavior to be learned is less readily observable 
and infrequently practiced. 

It should also be pointed out that our most effective modeling 
treatments involved an experimenter present to cue the subject on the 
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desired behavior. We think that this fact is important for the theo- 
retical development and understanding of observational learning. In 
earlier experimentation, the dependent variable, such as aggressive be- 
havior, may have been so dramatic, or so sharply contrasted with other 
on-going social behavior, that the observer had no difficulty detect- 
ing this behavior. However, trainees observing a complex verbal in- 
terchange on a complex set of ideas between a teacher and his students, 
may not easily or readily detect the specific form of the teacher's 
behavior which is occurring consistently. For example, a probing ques- 
tion may take many different forms, even though its general character- 
istics and timing remain the same. With an experimenter present to 
point this out, the trainee quickly learns what behavior to look for 
and what salient characteristics to identify. 

If this line of reasoning proves to be empirically sound, we will 
then move the study of observational learning to studies of the inter- 
action between characteristics of observational learning and character- 
istics of the behavior to be learned. In the study of teaching be- 
havior, this would mean that we would have to study what kinds of 
teacher behavior need to be cued when modeled and what kinds do not. 

Suggestions for Future Research; 

We are proceeding on a plan of research to study various condi- 
tions of modeling and reinforcement. It seems obvious that further 
p]fogress can be made by studying which characteristics of models make 
the modeling procedure more effective, and which characteristics of 

the feedback procedure make it more effective. For example, it is 
possible to present both positive and negative models of a desired 
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It is also possible to present both positive and negative feedback on 
the trainee's performance as he views it. Both of these are instances 
of characteristics of the modeling and feedback treatment which need 
to be investigated. 

The most significant line of research suggested by the results 
seems to be the study of the interactions between the kind of teaching 
behavior to be learned and the kind of treatment used to produce the 
behavior change. Although our studies do not demonstrate the fact, it 
seems apparent that some types of treatment are more likely to be ef- 
fective with specific kinds of teaching behavior and not with others. 

The first approach to the analysis of this problem will be to con- 
trast our most effective treatments across the series of dependent 
variables which we have developed. 

Still another line of potentially fruitful investigation would be 
to relate the characteristics of the trainee to the effectiveness of 
the treatment. We observed what appeared to be differential reactions 
to a common treatment. It was obvious that some subjects liked some 
treatments and some did not. How these differences may be related to 
the characteristics of the trainees is presently unknown. 

Studies need to be done on the behavior of the experimenter during 
an experimental treatment. We proceeded largely by applying common 
sense, using a standard form of verbal interaction to mediate our treat- 
ments. However, it may be that some experimenter behaviors interacted 
with the specifics of the treatment to render it more or less effective. 

It is also obvious that the personality characteristics of the ex- 
perimenter are themselves a set of variables which probably subtly in- 
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fluence the treatment. 



We do not know, for example, if male experimen- 
ters should be paired with male and female, or male alone, or female 
alone, trainees. (We also do not know whether or not the sex of a 

model has an effect on the modeling treatment) . 

Again on the basis of our naturalistic observations, we wonder if 
certain classes of teaching behavior as we have developed them can be, 
or will be, learned by both men and women trainees. The probi ng-style 
seems more 'natural' to a male teacher than to a female teacher. We 
may have been attempting to produce behavior changes which are less 
likely to be learned by women than by men. Although this is a problem 
to be studied, even if the results were to concur with the suggestion 
we have just made, the implication would not be that the female trainee 
should not acquire these teaching behaviors. The obvious problem for 
research is to determine how to change behaviors when attitudes or lack 
of previous experience makes it difficult for either men or women to 
learn professional behaviors. The study of emotional problems which 
may be associated with such learning is itself an interesting research 

problem. 

Implications for Training ; 

Although this research does not lead to detailed suggestions for 
specific modifications and developments in teacher training programs, 
the results are sufficiently solid to justify some general recommen- 
dations. First, it is clear that the general strategy underlying our 
experimentation provides a model for organizing training programs. We 
have been successful in teaching specific skills, although we have yet 
to demonstrate their general validity for producing pupil changes. 
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How6V6rj our prscticsl exporience is thst thoso skills genorslly tnsko 
sense to secondary teacher trainees, are relatively easy to teach, 
form the basis for extensive discussion on teaching problems, and are 
useful guides for continued supervision of trainees. We reiterate 
that the data supporting this statement is drawn from reflecting on 
our experience. 

We experienced relatively little difficulty in engaging the 
Stanford Secondary Education interns in these experiments. Such 
difficulties as we had were the results of failures on our part to 
provide adequate information, or misunderstandings that were easily 
remedied in successive experiments. The trainees generally liked the 
procedures, we had almost unanimous acceptance of the videotape prac- 
tices employed, and we felt that the trainees were pointed to a serious 
consideration of the elements of effective teaching. 

No great difficulty should be experienced in combining these 
skills, if one wishes to train for more complex behaviors. Similarly, 
the kinds of decisions that the use of these skills requires is fairly 
obvious. Discussing these decisions would broaden any program built 
upon a system of skill training. 

Our research does support the general rule that the use of feed- 
back procedures, comprised of self -viewing of one's videotaped teach- 
ing performance while being assisted by a supervisor (the experimenter 
in these studies) is a highly effective way of modifying teaching be- 
havior. The results of this training procedure were so systematically 
consistent that one could proceed with reasonable confidence that it 



would usually be effective and, in many cases, highly effective. 

We are less clear about recommending modeling procedures, al** 
though we are confident that in the long run such training procedures 
will prove to be effective. As noted above, the problem is to iden- 
tify those kinds of teaching skills for which modeling procedures are 
most effective. Until progress is made along this line of research, 
we cannot make specific suggestions about modeling techniques that 
might be used in teacher training programs. 

There is one exception, however, to this disclaimer. It seems 
reasonably clear, as an inference from the data of these experiments 
and from an analysis of our experience, that modeling procedures, how- 
ever they may be constructed, are likely to be highly effective only 
when a supervisor (again, the experimenter in these studies), cues the 
trainee on the desired behavior to be learned from observing the 
model . 

We are not claiming to have proved that the strategy used in 
conducting these experiments is the most effective way to train 
teachers. Certainly we have run no comparative programs to test such 
a conclusion. However, a moment's consideration will impress one with 
the large change that occurs in a teacher training program when exper- 
iments of this kind are introduced into it; when, in fact, they become 
a primary vehicle for a portion of the training. Having produced this 
change with reasonable acceptance on the part of out trainees, we are. 
confident that this experience points to the feasibility of making 
such training procedures a major part of a teacher training program. 
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Many problems remain to be resolved in utilizing such procedures 
and training programs. We have only preliminary information on the 
transfer effect of this training, information, however, which suggests 
a positive transfer effect. But our most powerful transfer studies 
remain to be done. We have not solved how one integrates training in 
decision**making with training on teaching skills simply because we 
have not had time to devote our attention to that complex problem. 

Nor have we yet demonstrated that these teaching skills effectively 
change pupil behavior in significant ways. We began simply: to dem- 

onstrate that teaching behavior could be learned if the conditions of 
learning were carefully controlled. We ourselves are engaged in con- 
tinued research in the problems cited here. We trust that our colleagues 
will find that this initial foray into the analysis of such complex be- 
haviors as teaching will stimulate further research in this domain. 
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Directions for Control Group (l) 



As you view your own perfonnsnce on this end the next three videotspesj 
try to determine your effectiveness in relation to: 

(1) Indications of the aims of the lesson. 

(2) Use of examples. 

(3) Effectiveness of teacher questions. 

(U) At' unt and spread of participation. 

(5) Pacing the lesson. 

(6) Teacher- student rapport. 



Directions given to Groups 2, 3» & 



PLEASE DO NOT DISCUSS THIS MATERIAL WITH OTHER INTERN TEACHERS UNTIL 
THE TAPING SESSIONS ARE COMPLETED IN DECEMBER# 

(Early in the Winter Quarter, the experiment, its purposes and results, 
will be fully discussed with you) . 

One important aspect of increasing your effectiveness in the class- 
room is concerned with achieving optimal student participation in the 
lesson. In this and the next three videotapes of your classroom per- 
formance, we would like you to try to increase student participation ija 

your lessons . 

One way of achieving this goal is to reward student responses when 
they occur. In general, a student will begin to respond more frequently 
if he is rewarded for doing so. If his responses are ignored or punished, 

participation will tend to drop off. 

Viewing Session One; In viewing your first tape, you are asked to: 

(1) Select five (5) students who in this lesson participated the least. 

To do this, use the attached analysis chart and put a check mark beside 
the Student's name every time he participates. After the tape is finished, 
select the five low participators. Since there may be more than 5 students 
who did not participate at all, select the five on the basis of alphabetical 

order . 

Your ob.lective over the next three tap i ng sessions will be to try to ge ^ 
these five students to participate more frequently. 

(2) Secondly, to help you analyze your ovn teacher behavior, you are 

also asked to: 

(a) count the number of times you rewarded students when they par- 
ticipated. 

(b) count the number of times you punished such responses. 

(c) count the number of times you ignored such responses. 
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NOTE; DO THIS ANALYSIS OF YOUR RESPONSES TO THE STUDENTS FOR EACH OF THE 
FOUR VIDEOTAPES (analysis charts are attached). 

A Student Participatory Response ; ( for your purposes of analysis) is any. 
verbal response. Do not count gestures as this would overly complicate 
your analysis. Once a student has responded, determine your reaction to 
his behavior, and classify it as rewarding, ignoring, or punitive. 

Teacher Responses ; In classifying your responses to student participation 
you should look for clear-cut responses. What your intentions are, and 
how you feel about a student are not relevant when you chart your reactions 
to his participatory behavior. Look for clear-cut teacher behavior , 
and classify your behavior as rewarding, ignoring or punitive. Do not 
classify your response on the basis of how you feel about a particular 

student . 

Scoring charts are attached to help you analyze your reactions to student 
participation. 



Appendix B 



I 



RATER CODE SHEET 
EXPERIMENT I 



i 



i 
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FORM II 



Tape Code No. 
Tape Segment 



Name of Intern 
Intern Code No 



Print 

• 



Number of Rater Sex of Intern _ 

Date of Rating Tape Session __ 

Grade Level Taping 



Rater Evaluation of Tape Quality! Sound — Video 



c c 
o o 
tn tn 
tn tn 

ss 

44 44 
O y 

0) U 
0. B 
>» •!-< 
H H 

Type 

and 

Time 


4J 
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0 
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0 
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0) 

CO 

M 

or 
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4J 
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4J O ->-• 

u c 
0) 0 
>■* ^ 

o > 
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® a Timing of Reinforcement 
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Reinforcement 
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0) 
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CO 
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Rs,Ci,Su 
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r* M M o Q 

® - Student Response 
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2 


3 


4 
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6 
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8 
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I.S. 
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Column 



Date Information: Sheet One (Pencil) 



Expected Symbols 



1 

2 

3 

5 

»» 6 
»» Y 
8 

** 9 i 
*»10 j 
»»11 1 
»»12 I 




i 

18 

*19 

*20 

*21 

*22 

*23 

*2k 

*25 

*26 

27 

28 

29 



30 

31 

32 

33 
3U 
35 



36 

37 

38 

39 

ho 

Ul 



Tape code number (l - 210) 

Tape segment letter (A - D) 

Treatment code number (l to U) 

Experimental session number (l to U) 

Sex of Intern 

Age of Intern 

Grade level taught 

Date of taping 

Date of playback 

Difference score (3 scores here) 

School lesson taped at 

Home state of Intern 

Tape sound quality RATER (-3 to +3) 

Tape picture quality RATER (-3 to +3) 

Tape sound quality INTERN (-3 to +3) 

Tape picture quality INTERN (-3 to +3) 

Raters' names (coded by last names) 

Date of rating 

Questionnaire 01 - increased teaching effectiveness (-3 to +3) 
Questionnaire 02 - tapes helpful (-3 to +3) 

Questionnaire 03 - supervision helpful (-3 to +3) 

Questionnaire 0k - viewing becomes boring (-3 to +3) 

Questionnaire <5*12 - 20 minute sessions too long (yes or no) 
Questionnaire <5*12 - 20 minute sessions just right (yes or no) 
Questionnaire <5*12 - 20 minute sessions too short (yes or no) 
Questionnaire <5*13 - Useful to see others' tapes (yes or no) 

Tape time lecture-discussion (LD) + minutes & tenths of minutes 

Tape time Group Work ( GW )+ minutes & tenths of minutes 

Tape t im e Individual Study (IS) + minutes and tenths of minutes: 

T reads to class and CO 

COLUMN 2 & 3 

Total number of male responders 
Total number female responders 
Total number male responses 
Total number female responses 
Total number of slashes 

Total number of responses (inculdes slashes): Add Col. 32, 33 ^ 3^ 

COLUMN U 

Number of D S I male 
Number of D S I female 
Number of D S I total 
Number of D S G t otal 
Number of D S G-I male 
Number of D S G-I female 
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1 



Column 



h2 

h3 

UU 

U5 

U6 

h7 

kd 

k9 

50 

51 

52 



53 

55 

56 

57 

58 

59 

60 

61 

62 
63 
6k 

65 

66 
67 



Date Information; Sheet One (Pencil) Expected Symbols 

Number of D S G-I total 

Number of D S G-V male, also D S G-V-I 

Number of D S G-V female, also D S G-V-I 

Number of D S G-V total , also D S G-V-I, also D S G-V where no 
M or F indicated 
Number of D S S tot al 
Number of V male 
Number of V female 

Number of V total, also Vs where no M or F indicated 
Number of V? male 
Number of V? female 

Number of V? total, also V?'s where no M or F indicated 



COLUMN 5 

Time of reinforcement (B) male 
Time of reinforcement (B) female 
Time of reinforcement (B) total 
Time of reinforcement (D) male 
Time of reinforcement (D) female 
Time of reinforcement (D) total 
Time of reinforcement (A) male 
Time of reinforcement (A) female 
Time of reinforcement (A) total. 

Number of (+) affect loading male 
Number of (+) affect loading female 
Number of (+) affect loading total 

Number of 0 affect loading male; Add col. 53, 56, 59 » subtract 

col. 62 -u^ 4. 

Number of 0 affect loading female: Add col. 5^» 57, 60, subtract 

col. 63 ,, 

Number of 0 affect loading total; Add col 65 & o6 
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Summary Sheet Breakdown 



Page 3 



Column 




Data Information! Sheet Two (Red) Expected Symbols 



COLUMN 5 

Number of Nods and Smiles (N S) total , or N or S 
COLUMN 6 

Number of Moves toward (^) total 
Number of Moves away ) total 
Number of Eyes off (OFF) total 
Number of Frowns (F) total 



COLUMN T 

Number of statements including word Good total 
Number of right (R) total , also T.R., Y.R., (Not A.R.) 

Number of fine (F) tota^, also TaFa 
Number of excellent (E) total , also Ex a 
Number of Okay (OK) total , also all right AaRa 
Number of Um-Hmm (UH) total 

Number of Yah or Yes (Y) total « j.m \ 

Number of +Names or +Call's attention to ^ a l, (+N, +Ca, 

Other positive verbal reinforcement total , laea, TaYa, i.ra, Wa, 
Verbal overflow (+) statements total 
No a wrong (X) total 



(T-), (B-), (PP-), (NR-). 



[legative naming (-Ca, -N) total 
IVll negative statements total, i.e, 

(S— ) , etc a / \ X T 

Verbal negative overflow (-} tot_^ t i o 

Grand total positive non-verbal reinforcement, add col. 1,2, Ue ) 

Grand total negative non-verbal reinforcement, add col. 3,4,5. 

Grand total positive verbal reinforcement, add col. ° ^ 

Grand total negative verbal reinforcement, add col. 16 to 19 tor 

l6, 1T^ 18 an<i 19) 

Teacher No response (NR or NC) males 



COLUMN 8 

Teacher No response (NR or NC) females 
Teacher No response (NR or NC) total 
Teacher Response Style Redirect (Rd) total 
Teacher Response Style Refocus (Rf) total 
Teacher Response Style Rephrase (Rp) total 
Teacher Response Style Repeat (R2) total 
Teacher Response Style Clarification (Cl) to t al 
Teacher Response Style Summarize (Su) tota^ 
Teacher Response Style T feedback (TF) total 
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Summary Sheet Breakdown 



Page ^ 



Column. 



3k 

35 

36 
3T 

38 

39 
UO 



Ul 

k2 

UU 

k3 

he 

»»U8 



o 

ERIC 



nat.a. Information: Sheet Two (Red) Expected Symboj^ 

COLUMN 8 

Teacher Response Style Qualified Reinforcement (QR, or +Q) t£t|^ 
Teacher Response Style Qualified Reinforcement (-(^, or -QJ to^ 
Teacher Response Style Post-Hoc Reinforcement (PR) tpt_^ 

Teacher Response Style Role Play (RP) tota^ 

Teacher Response Style T clarifies (TC) total 
Teacher Response Style Blackboard (+BB) total 
Teacher Response Style Blackboard ,-BB) total 

COLUMN 9 

Student Responses No Group Response (OG) total 
Student Responses No Individual Response (01) tot^ 

Student Responses Chorus answer (Ch) total 
Student Responses Irrelevant Response (10) total 
Student Responses Interrupt Negative (I-) total 
Student Responses laugh at T (LT) total 
Student Responses laugh at S (LP) total 
Subject matter major of intern 
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Appendix D 



RATING DI REACTIONS 
EXPERIMENT I 



I 
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I. General Information; 



Tape Session 

Intern Code No. 

(Rater leave this blank) 



1. Name of Intern (PRINT “““e) 

2. Name of Rater Date of Rating — 

3. Sex of Intern (circle one) M F 

U. Tape Code: (AA to ZZ) (on tape box) 

5. Tape Segment (circle one): 1st 2nd 3rd Uth 

6. Rater evaluation of tape quality: 



+3: I agree very much -3: I disagree very much 

+2: I agree on the whole -2: I disagree on the whole 

+1: I agree a little -1: I disagree a little 

a. The sound quality of the tapes was good enough to permit me to 

adequately analyze the performance of the intern: 

b. The video (picture) quality of the tape was good enough to permit 

me to adequately analyze the performance of the intern. 



II. Tape Analysis 

Column I: Tyne of lesson and Time : (l) In most cases, rate this 

at the end of the lesson. Most lessons will be predominately lecture 
and discussion (L.D.), and will run for 20 minutes. Thus, code them as 
L.D.-20. (2) In some cases teachers will break the class up into small 
groups for project and small group discussion for part or all of the 
tape. When this happens, record as G.W. (for group work), and indicate 
time spent, e.g., G.W. -10. When you have both L.D. and G.W., code them 
both and indicate exact time, e.g., G.W. -11, L.D, -9. (3) In a few 

cases there will be desk work such as tests, reading, homework, where 
no T-P interaction occurs. Code this as I.S., and indicate time, e.g., 



Teacher spends first 5 minutes of tape-time giving P's a reading assign- 
ment and then discussing the assignment with the class. Code as I.S.-5, 



L.D. -15. 
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Summary : L . D . : 


Lecture/discussion 


G.W.: 


Group Work 


I.S.: 


Individual study 



NOTE: Run each tape through to the finish and record exact time --not 

all tapes are 20 minutes. 

Colum^^: Student Number : The pxirpose of this column is to find out how 

mauiy student responses (toted number) occur during the tape. A response 
is any uninterrupted vocal response by a student. If a student and teacher 
have a verbal exchange • his responses will be recorded as follows: 



S: 


"What time is it?" 


1 


T: 


"5:00 P.M.." 




S: 


"Are you sure?" 


1 


T: 


"Yes . " 




S: 


"That is late." 


1 



On the example, 3 student responses have occurred, and all of them were 
made by student number one. Scored this way we can get total number of 
S responses, and how many times each S responds by adding up the column 
of figures. 

Coiiinm Pnpil»9 Sex : In all but a few cases you can identify this 

easily. Score as M or F. 

nniiinin U; Direct Solicit or Volunteer (P.S. or V. ) : In direct solicit . 

T asks P a question. In volunteer. P volunteers a comment without being 

asked by T. On direct solicit by T, code this as: 

D.S. I: D.S. of individual pupil 

D.S. G: D.S. of group response 

Colxmin 5 : Pupil No Response ; T asks P a question and he does not answer. 

Code this simply by a stroke (/) . If T asks a general question of the 
class, and there is no response, code it as (C). 



Column 6: Timing of T Reinforcement: 



1. T reinforces S before he begins to speak (B) 

2. T reinforces S during his speech (D) 

3. T reinforces S after his speech (a) 

Uncertain or unknown (U) 

Column T ; Affect Loading of T Reinforcement ; The word "good" (or any 
other reinforcement) can be said by T in a negative, neutral or positive 
manner. If T reinforces with enthusiasm, the affect loading (emotional 
overtone) is positive, so code it (+). If neutral code it (O). If it 
is obviously negative, punitive or sarcastic, score it (-). 



Column 8: T Verbal POSITIVE Reinforcement: T uses words such as: 



1. 


Good, or very good: (V.G.) 


T. 


Fine: (F) 


2. 


Good point: (G.P. ) 


8. 


Umm-hmm: (U.H.) 


3. 


Interesting point (I.P.) 


9. 


Good beginning: (G.B.) 


k. 


Good question: (G.Q.) 
Right: (R) 

Okay: (O.K.) 


10. 


Calls class attention to S 
positively: (C.A.); e.g.. 


5. 

6. 




"Listen! John has a good point. 



In general, use the first letter of the word in coding. You will tend to 
be getting combinations such as T says "good", then nods and smiles; 
code this as G., N.S.. T may say, "Good, fine. That's fine! It was a 
good question." Code this as G., F., G.Q.. 



NOTE : If this proves to be impossible in actual rating, we will simply 

use a stroke (/) to set the number of such terms. 



Column 9 : Urges and Encourages : T says to S during S's answer such 

things as: "Yes", "Go on", "Um-hm", etc.. Use a stroke (/) for each 

T response unit. 

Column 10 : T Negative Verbal Reinforcement : Here T will: 

1. Threaten Student (T) "if you do that again " 

2. Belittle Student (B) 

3. Recall past punishments (RP): "Remember that once before... 
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4. Require task to be done over (DO) 

5. Impose physical restriction (PR): "Sit down and shut up!" 

6. Sand student out of room (0): "Go to the office I" 

7. Withdraw privileges (WP): "No free periods because you were bad." 

8. Recall past instructions negatively (NR): "Had you paid atten- 

tion you would have known that.: 

9. Call attention to inappropriate behavior (CA) : Here T focuses 

class on, for example, John's "Bad" behavior. 

10. Give negative command (NC) : "Get to work and be quiet." 

11. "Name" the student in a warning and negative manner (Na): "John. .. 

Column 11: T NON-VERBAL Positive Reinforcement: Instead of (and in 

combination with) + words, T may: 

1. Nod and smile (N & S) 

2. T moves towards S 

3. T keeps eyes on S (on) as S is answering or going to answer a 
question. 

Column 12: T NON-VERBAL Negative Reinforcement: 



1. T scowls, frowns (F) 

2. T moves away from S 

3. T takes eyes off S while S is speaking, (off) 

Column 13 : T Informative Feedback : In response to a S's question or 

commenTT"! gives the answer without commenting on S's comment. 

e.g., S: "What is the time?" 

T: "3 P.M." 

NOTE: Code T response of "No," "Wrong," "Incorrect" here as: X 

Column 14: T N o Response : T does not respond to S but makes no comment 

or uhaiigg s the subject, goes back to the board, etc. Code by stroke (/). 

Column 15 : T Repeat : T repeats S's answer: 

1. R 0 - T repeats S's answer only — does not comment on it. 

2. R + - T repeats S's answer and says "good", for example. This • 

would be coded R-G 
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Column 16 ! T Techniques ! Redirects (Re) : Here T changes direction of 

participation by turning to another P and saying: 

1 . Redirects 

1) "Do you agree with that?" 

2) "Can you add to that?" 

3) "Can you help him (us) out here?" 

4) "How do you feel about that?" 

2. Teacher Qualified Reinforcement : 

1) Positive (-^0) : Here T tells P he is partly correct, 

then qualifies: e.g , 

a. "Good as far as you have gone, but ..." 

b. "You are on the right track, but ..." 

c. "Fine, however ..." 

d. "Right, but don't forget . . ." 

2) Negative (-0): as above, except negative. 

a. "That's foolish, you forget . . ." 

b. "Of course not. Anyone knows that ..." 

c. "No'. Don't you know that ..." 

3* Post hoc Pupil Reinforcement (PR): T, while discussing 

a point says such things as: "John made a good point on 

this earlier." 

4. Role Play (RP) : T asks class to "put yourself" in the 

situation; e.g., "Suppose you were a fireman ... on a ^ 
desert island ... in charge of the country . . . etc." 

5. Chorus Answers: the Class answers in chorus (Ch) 
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INTERN QUESTIONNAIRE RESULTS 
EXPERIMENT I 
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Question One : "The opportunity to view the four tapes of my own teaching 



increased my effectiveness in the classroom," 

Question Two : "The supervision I received during the tapes was helpful 

in increasing my classroom effectiveness." 

Question Three : "In general, viewing the four tapes was an enjoyable ex- 

perience for me," 

Question Four : "After some time viewing one's own performance by video- 

tape becomes boring." 







N» 


Question 

1 


Question 

2 


Question 

3 


Question 

k 


I, 


Controls 


15 


1.93 


-1.92 


2.00 


0 


II 0 


Self-Reinforcement 


13 


1.15 


-1.U6 


1.92 


+0.31 


III 


. Minimal Supervision 


Ih 


1.78 


+1.35 


1.57 


-0.14 


IV. 


Maximal Supervision 


12 


1.91 


+2.66 


1.91 


-2.00 



*note: Did not get questionnaire results from all S's in the study. 

Complete sampling would have yielded 18 from each experimental group. 

Question Seven : "We would appreciate your criticisms and comments on the 

taping sessions." 

Question Eight ; "We would appreciate your suggestions for improving future 
taping sessions similar to those just carried out," 

Question Nine : "We would appreciate your suggestions for making supervision 

more effect iveo" 

The following thematic breakdown is for group U (maximiam supervision). 
Questions seven and nine are combined since there was extensive overlap 
in the content of intern responses . 

Summary of questions T and 9 for group U ; Frequency of Mention 

1. More time should be allowed for each playback to 1 

permit more discussion between intern and supervisor. 
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Siinimary of questions T and 9 for Grou^ h (Cont*d) 



Frequency of Mention 



A 



2, More sessions like those in the experiment are 
needed. 

3. The supervision I received during the session has been 
the most valuable feature of the intern program to date 

U. The supervision during the sessions was very helpful 
( and/or extremely valuable ) . 

5. The supervision was generally helpful. 

6. Supervisor approach was "a bit pressured and bull- 
dozing." 

T. Supervisor's remarks were "too planned" so need more 
time in playbacks. 

8. Supervisor should not censure the intern. 

SuTTMTiarv of Question 8 (Group 

1. More advanced warning as to when we will be taped. 

2. Sessions should be more spread out over time. 

3. All interns should receive this supervision. 

U, Supervision should be integrated into the intern pro- 
gram more fully, and Stanford supervisor and E work 
together . 

Rnmma.ry of Questions T and 9 (Group 3) 

1. Supervision was most (very) helpful. 

2. Supervision was helpful (worthwhile). 

3. More supervision and guidance needed of broader 
scope and more extensive. 

k. Would prefer experienced teacher rather than graduate 
student in psychology for supervision 

Surnmarv of Question 8 (Group 3) 

l, Pupils were distracted by taping sessions. 

2. There vjere too many tapes. 

3. More tapes are needed. 

U. More time in playbacks needed. 

5„ Integrate the sessions into the intern program. 

6. More advanced warning as to when we will be taped. 
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3 

k 

6 

1 

1 

1 

1 

5 

2 

2 

1 

5 

3 

6 

1 

3 

2 

3 

2 

2 

3 









Frequency of Mention 
k 



Summary of Question 8 (Group 3) Cont*d 

T. Sessions should be more spread out over time 

8. When found out it was an experiment was very- 
displeased. 



3 



-181- 







mm 



1 



Appendix F 



INTERN INFORMATION 
EXPERIMENT II 
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During the fall we were trying out various patterns of supervision 
to determine how best to use supervision with TV recording. This series 
we called Experiment if^l. Some of you received additional supervision and 
some did not. Now we are changing the variable we are working with, and 
everyone will receive the same amount and type of supervision. We will 
be varying the time of taping and playbacks instead of style of supervi- 
sion. 

In the fall the goal was to increase the amount of pupil participa- 
tion. A full report on results is forthcoming. The results are now being 
analyzed. This spring, in the recording series which we are calling 
Experiment #2, you will have an opportunity to develop specific skills 
that will help you upgrade the quality of student responses. The intern 
or teacher behavior of interest in this experiment is redirecting and re- 
focussing pupils ' questions and statements. ^ prompt their self-correction 
and renewed attempts at answers. This is a useful skii.1 for the intern 
to acquire, for it allows him to control classroom interaction in such a 
way that pupils may develop increased critical self-awareness, more clear- 
ly analyze the implications of their own and others' questions and con- 
clusions, facilitate the transition from relatively superficial to more 
complex levels of analysis of a problem, and to facilitate transfer. 

These in general are to be your goals during the coming weeks as you 
are videotaped. The videotape provides all of us with an excellent means 
for practicing and mastering these skills. Some minor restrictions on 
your lessons and time for playbacks are more than offset by the potential 
gains in professional skill. 

The objective of the current study is to help interns develop more 
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alternatives in responding to student questions and statements. 

BASIC RULE I ; The Teacher (T) does not give immediate answers to pupils' 
questions. 

Specific Teacher Techniques (where appropriate) 

1. Ask the pupil (P) to break his question down into more 
easily answered parts. 

2. T rephrases P question without giving the answer (this is 

a technique developed by Carl Rogers-- the idea here is to 
rephrase the question in such a way as to elicit further 
clarification by P) . 

3. T may say, "What things (or aspects of the problem) must 
we consider in order to get the solution; i.e., where 

do we look for the answer?" 

4. T redirects the P's question or statement to another P or 
P's by saying such things as: 

(a) "Can you help him out?" 

(b) "Do you agree with that?" 

(c) "Can you add to that?" 

(d) "What are the implications of this for . . .?" 

(e) "Can you summarize this for us?" 

5. Role Play : P may ask what the reasons are for doing some- 

thing in a particular way (e.g., P asks T, "Why did Germany 
attack Poland before France?"). T^s response here is to say, 
"All right, I want you to be that person (or in the situa- 
tion for a moment. How do you justify the action (method) 
you have taken?" 
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T Qualified Response ; Assuming P has no idea of the answer 



(after probing by T) , T may give part of the answer to P 
and require him to find a solution and report back to 
class (or a legitimate question for group work later on) . 

7. Encouragement of Alternatives ; In response to a P question 
or statement T may say, "Before we try to reach a conclu- 
sion (or answer) here, let us explore as many avenues as 
possible. Think of as many (answers, reasons, solutions) 
as you can." Encourage your students to divergent thinking. 
BASIC RULE II ; T does not immediately accept P's answer or statement. 
Possible T techniques : 

1. Clarification; T asks P to rephrase or clarify his answer 
or statement . 

2. Summarize ; T asks P to summarize his answer, statement, or 
class discussion. 

3. Critical Awareness ; T asks P (or class in general) such 
things as; 

a) "What are we assuming here?" 

b) "Is this one question or several questions?" 

c) "Can we break the problem down into more manageable 

parts for analysis?" 

d) "Have we (you) oversimplified this -■ is there more to 

it?" 

e) "How would someone who took the opposite view respond 

to this?" 

4. Redirec t; Here, as outlined earlier, T changes the 
direction of participation to another pupil and says; 

a) "Do you agree?" 

b) " . . . add to that?" 

c) "What else might be said?" 

5. Refocus ; T refocusses discussion or class attention on 
another question or problem (usually a related one, but 
not necessarily so) -- T may say; 
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a) "Good'. What are the implications of this for . . .. 

b) "Let's look at it this way . . .jwhat is your answer 

on 

c) "How does this relate to ... • 

As you can see. these specific techniques which are appropriate for 
various situations are inappropriate in others. In preparing your les- 
sons for TV, do all you can to allow yourself the opportunity to practice 
these techniques with the class. In the playback sessions, we will pro- 
vide additional instruction and specifically relate the above-nan«d and 
other techniques to your particular situation. In general, discussion- 
type lessons, with material that allows you and the class to interact 
and analyse problem areas will be best. Please avoid individual seat 
work, tests, quizses, and group work on taping days. 

vACTr pm.E TTl ! As the students start moving toward the goals you have 
set. rhese desirable responses . At first their attempts may be 

far short of the ideal: to get them to approach that ideal, reinforce or 
reward them a^ each stgE along the wag. Don’t wait for the perfect answer 
- reward each effort so that they move forward bit by bit. 



GENERAL INFORMATION: 

1 Interns will generally be given at least one week’s notice ££ior 
to taping dates. We are anxious to avoid the mistakes we made 
in this Regard in the fall study. There are some situations 
which are beyond our control - we ask tha t you recognize the 
possibility of (a) machine breakdowns (b) faulty videotapes 
(c) interns missing playbacks. If this happens, you "ill ^e 
notified promptly and the taping will be rescheduled. Most of 
these bugs have been ironed out so that we really don t expec 
holdups of this kind (we have more machines now so scheduling 

is facilitated). 

2. As usual, you will be taped for 20 minutes on each 

lessons. You will be asked to see these tapes in playba 

sessions . 
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Appendix G 



DIRECTIONS FOR INTERN TEACHERS 



EXPERIMENT III 



Following the pretest, each subject received these directiore 

in Step Two of the general procedure. The two Symbolic Modeling 

groups reread the instructions in Steps Four and Eight. 

DIRECTIONS FOR INTERNS: BASIC QUESTIONING TECHNIQUES 

READ THE FOLLOWING INSTRUCTIONS VERY CAREFULLY 

BEFORE VIEWING THE VIDEOTAPE 

Today you will have an opportunity to develop skills in basic 
classroom questioning techniques. The session is designed to help 
you extend the range and quality of your questioning techniques in 
such a way that the pupils you teach are led to think more deeply 

about problems raised in class. 

The techniques outlined below are designed to be used in dis- 
cussion, review, and inductively organized lessons where active pupil 
participation is prerequisite to the realization of the goals of in- 
struction. Any given technique may be appropriate in one situation 
but not in another. The selection of a particular technique depends 
upon the extent to which, in your judgment, it requires the pupil to 
analyze critically a problem or justify rationally his answer. Do 
not use a given technique unless you feel it contributes to the ed- 
ucational relevance of the lesson. 

YOUR GOAL IS TO ASK PENETRATING AND PROBING QUESTIONS THAT REQUIRE 

PUPILS TO GO BEYOND SUPERFICIAL. "FIRST -ANS WER” RESPONSES. 

BASIC QUESTIONING TECHNIQUES: There are two ways of achieving the 

above goal: 1) The teacher asks penetrating questions that require 

pupils to get the heart of the problem. This forestalls superficial 
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answers. Wliether or not you are able to do this depends largely upon 
your knowledge of relevant content. 2) The second approach is based on 
specific techniques that may be used after the pupil has responded in 
some way (i.e., a question, a comment, an answer to a teacher question). 

The goal here is to get the pupil to go beyond his first response. You 
attempting to produce greater critical awareness and depth by Probing . 

Your cue is the pupil's response--once it has occurred, don't immed- 
iately go on with the discussion yourself. Probe his answer by means 
of one of the techniques outlined below. 

I. Teacher Seeks Further Clarification by the Pupil ; You 
may ask the pupil for more information and/or more meaning. You may re- 
spond to the pupil's responses by saying such things as: 

a) "What do you mean? " 

b) "Please rephrase/clarify what you mean." 

c) "Can you explain that further?" 

d) "What do you mean by the term . . .?" 

jj , Teacher Seeks Increased Pupil Critical Awareness ; Here 
you are requiring the pupil to justify rationally his response. You 

may say; , . «.i 

a) "What are you/we assuming here? 

b) "Why do you think that is so?" 

c) "Have we/you oversimplified the issue--is there more to it?" 

d) "Is this one or several questions?" 

e) "How would someone who took the opposite point of view 

respond to this?" 

III. Teacher Seeks to Refocus the Pupil Response ; If a pupil 
has given a high quality answer, it might seem unnecessary to Probe it. 
However, you could Refocus his or the class's attention on a related issue. 

a) "Good! What are the implications of this for . . .?" 

b) "How does this relate to . . .?" 



- 189 - 






c) "Can you take it from there and tie it in to . . .?" 

IV. Teacher Prompts Pupil ; In Prompting , you are giving 
the pupil a hint to help h£m go on and answer a question. Suppos- 
ing a pupil has given an, "I don't know", or, "I'm not sure" type of 
response. Rather than giving him the answer or redirecting the ques- 
tion to another pupil, you may give the puzzled student a hint. 

e.g.. Teacher: "John, define the term polygenesis." 

John: "I can't do it." 

Teacher: (prompt) "What does poly mean?" or, "Well, 

means origin or birth, and poly means . . ?" 
This technique allows you to Probe even though at first, it appears 
that the pupil can't answer the question. 

V. Redirect : This is not Probing technique per se . It 
helps you bring other students into the discussion quickly while still 
using Probing techniques. In Redirect , you merely change the direction 
of interaction from yourself and pupil (one) to yourself and another 
pupil (two). 

e.g.. Teacher: "What is the relationship between pressure 

and volume?" 

p^: "As pressure goes up, the gas is condensed." 

Teacher: (to p^) "Can you tell us what is meant by con- 
densed?" Or, "Can you restate that in terms 
of volume, p ?" 

To sum up, the techniques outlined above have two things in common: 

1) They are initiated by the teacher immediately after the 
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pupil has responded. 

2) They require the pupil to go beyond the information he 
has already given. 

CONCLUDING REMARKS: Try to use the techniques as frequently as you 

can. Do not stay with one given technique for too long at one time. 

In addition, don't forget to reinforce when you Probe-«there might be 
a tendency at first to behave like a "Philadelphia lawyer." 

If you prefer to run through the first five-minute lesson 
as a warmup, this would be fine. You may teach the same lesson over 
two or three times. We will focus more on Probing than on transmitting 
new or complex material. The maximum amount of time for the session 

will be two hours. 

You will be working with ninth and tenth grade pupils who 
have had some experience with modeling sessions and who will be very 

cooperative and responsive. 

Thank you for your help. 
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FOR TRAINEES IN EXPERIMENTAL 
CONDITIONS USING VIDEO-TAPED TEACHING 
PERFORMANCES OF THMSELVES OR OF MODELS 

The following instructions were added for these trainees: 

With these ideas in mind, you may now critically view your own 
performance. Other parts or segments of the session will give you an 
opportunity to practice and reflect upon these techniques. 

One -l ast note : In addition to giving you training in basic ques- 

tioning techniques, we are trying to determine the best way to teach 
them. We are using several training methods. All of them have been 
shown to be effective, but we want to find out which particular method 
is most effective. Thus, different groups of teachers will be taught the 
same thing in different ways. 

Please do not discuss any part of your training with other teachers until 
the quarter is over . 

We will review the techniques and give you the results once the data are 
analyzed. 



Appendix H 



THE RATER'S MANUAL 



EXPERIMENT III 



THE RATER'S MANUAL 



The Rater's Manual served as the basic training device in develop- 
ing rating skills. It will he noted that the manual is keyed to The 
Rater's Code Sheet. The latter was used to record the relevant behaviors 
as the tape was viewed by the rater. The reader may find it helpful to 
refer to the Code Sheet before reading the manual. 

Note that Columns (l), (3), (13), ilh) and (15) are not discussed 
in the Directions for Interns . Column (l) served a dual function. First, 
it was found that by requiring the raters to record these behaviors, 
reliability was increased. Raters were forced to make finer distinctions 
between Prompting and non-pupil-response contingent teacher verbal behav- 
ior. In addition. Columns (l) and (lU) served to provide an index of 
the generalization of Probing techniques to other areas of teacher verbal 
behavior. 

Column (3) was included to see whether or not the intern's rate of 
reinforcement dropped off with an increase in Probing . The - two techni- 
ques are deemed by the investigator to be complimentary rather than dis- 
crete strategies. In fact. Probing grew out of an earlier investigation 
where reinforcement was used to increase pupil participation. Probing 
was developed to increase the quality of such participation. 

Columns (13) and (15) constitute relatively low-frequency Probing 
techniques. They were not included in the written instructions to the 
interns as it was felt that a plethora of specifics would tend to increase 
interference. The models were, however, instructed to demonstrate both of 
these techniques at least once in their tapes. 

The use of each of the latter techniques was pointed out to the 
intern in discrimination training phases of the experiment. However, 
the experimenters did not single them out for special attention. 
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THE RATER ‘S MANUAL 

GENERAL INFORMATION : 

a) Tape Box Number 

b ) Segment rated from to__ 

c) Rater Evaluation of Tape 
Quality: 

a. sound 

b. video 



d) Name of Rater and date 

Type of Lesson and Time: 

L.D, 

I.S. 

G.W. 

¥ 

Raters are responsible for filling in all of the above data. The bulk 
of the information is to be gathered after the tape has been rated. 

Print each bit of information as clarity is essential. 

a) Tape Box Number : Ask operator for this number before you begin to ^ 

ratO} and before it is put on the machine, if possible. 

b) Segment Rated : Get precise beginning and ending times (accurate to 

1/10 of a minute) of the segment of tape rated. This should be read 
off the machine by the operator and checked also against the infor- 
mation on the box. 

c) Rater Evaluation of Tape Quality : 

+3-1 agree very much -3-1 disagree very much 

+2-1 agree on the whole -2-1 disagree on the whole 

+1 - I agree a little —1 ~ 1 disagree a little 

1) The sound quality of the tapes was good enough to permit me to 

A 

adequately analyze the performance of the intern: 

2) The video (picture) quality of the tape was good enough to 
permit me to adequately analyze the performance of the 

i 

intern: 



e) Name of Intern 

f) Sex of Intern 

g) Tape Session No. 

h) Date of Taping 

i) Subject Matter of Lesson 
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3) 


Name of Intern: 


write 


in 


after 


tape 


is 


h) 


Sex of Intern: 


tt 


tt 


tt 


tt 


tt 


5) 


Tape Session No: 


tt 


tt 


tt 


tt 


tt 


6) 


Date of Taping: 


tt 


tt 


tt 


tt 


tt 


7) 


Sub.lect Matter of 
Lesson: 


tt 


tt 


tt 


tt 


tt 



ft 



tt 

tt 



tt 



TYPE OF LESSON AND TIME ; Rate this (as a rule) at the end of the lesson. 
Most lessons will he five-minute Lecture-Discussion (L.D.) lessons. Code 
them on the L.D. line as 5.0. In some cases, I may have group work (G.W.) 
or individual study (l.S.) for part or all of the lesson. If this happens, 
code appropriately. 

Lecture-Discussion : (L.D.) 

(G.S.). 



Group Work: 



Individual Study (a test, silent reading, etc.): 

(I.S.). 



INTRODUCTION TO RATING : 
classroom interaction. 



Be sure to run each tape through to finish and record exact time. 

The code sheet is based on a "typical” unit of 
Usually: (l) T asks a question, 

then: (2) P responds. 

then: (3) T Probes or does not Probe this 

pupil response. Our primary interest is in T’s response. Does T Probe 
or Not Probe? Every unit of T-P interaction is to be coded this way . 



COLUMN ONE: T REPHRASES OR CLARIFIES HIS OWN RESPONSE; While T’s goal 



is to get pupils to Clarify and Rephrase their responses , this may spread 
to the teacher's responses as well. In asking questions, T may Clarify 
or Rephrase it. e.g., T: "What is ironical about this situation?" 

T: "That is, are our expectations about what 

should happen reversed here? " 

In the above example T asks a question, then clarifies it by becoming 
more precise in his definition of "ironical." Frequently, T rephrases 
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on a more obvious level ^ e.g.f 

T: "Define ' economic ' . " 

T: "What do we mean by economics?" 

It will be noted that column (l) (T clarifies or rephrases own response) 
is similar in some ways to column {ll)% Prompting. The key difference 
is that in Prompt ing> T is responding to a pupil response ^ whereas in 
column (1), he is responding to his own prior response with no inter^ 
vening pupil response having occurred . Unless the pupil actually responds, 

T is classed as in column one. 



Example of T rephrasing/clarifying 
his own response 
COLUMN (1) 

T: "Define the word ironic . " . . . . 

What do we mean when we say: 
'that was ironic'?" 



Example of T prompting 
COLUMN (11) 

"Define the word ironic . " 

"I'm not sure." 

(to p) "It has something to 
do with a reversal of our 
expectations— does that help? 
Can you give me an example of 
an ironical situation?" 



T: 

P: 

T: 



COLUMN TWO ; PUPIL RESPONSE ; P may respond in one of five ways: 

1) Comment (C) 

2) Question (Q) 

3) Answer (<V) 

k) No Response ; T asks a question, P does not respond. 

5 ) Bhetorlcal Comment/Q.uestlon (C? ) e.g. , T has asked for possible 
uses of petroleum: 

(both may P: "Gasoline would be one use — wouldn't it?" 

be defined P: "Wouldn't paving be another use?" 

as C? ) 
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COLUMN THREE; 



T REINFORCES PUPIL; 



T may reward a P response by saying: 



"Good,” "Fine," "Excellent," "Right," "Yes," "Correct," etc. If T 
uses verbal reinforcement as outlined above, then record this in column 3. 
Count each bit of reinforcement as a (l) 



e.g.. 



T Response No, of Reinforcements 

T: "Good!" 1 

T: "Good! That's a good 2 

answer, John." 

T: "Correct. In fact, quite ^ 

good. You know your stuff." 



COLUMN FOUR : T REPEATS PUPIL RESPONSE ; This column covers repetition and 

simple rephrase as when T does not add anything to the gist of a P response 



in repeating it, but does reword it slightly. Note that T may or may not 
Probe once he has reinforced and/or repeated a P response. 



NO PROBE AND PROBE COLUMNS : These two half columns permit you to indicate 

whether or not T has Probed a P response. If T has accepted (or No-Probed ) 
a P response then complete the (X) in the NO PROBE half-column. If T has 
Probed the P response, then leave the NO PROBE column blank and complete 
the (X) in the PROBE half-column. Every T response following a P response 
must be coded as a Probe or No— Probe response . Once coded, each type of 
response must be further classified as some particular type of No-Probe 
(see columns 5 and 6) or Probe (see columns 7 - 12). 

A T response is classified as a Probe when, once the pupil has 
responded, T asks that pupil for further comment on his response. There 
are five types of T Probing behavior (see columns 7 - il). A No-Probe T 
response occurs when T does not ask for more information, meaning or cog- 
nitive activity from P that is based on P's response. Instead T may: 

1 ) ignore the response (col. 6) 

2 ) reinforce only (col. 3) 

3 ) repeat only (col. 5) 

U) add more information himself (col. 5) 

5 ) or simply tell P his answer is wrong (col. 6) 
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COLUMN FIVEt T ANSWERS P’s QUESTION OR ADDS MORE INFORMATION 



This is one type of NO-PROBE response, 
e.g., (T answers question) 

P: "Who discovered America?" 

T: "Good question. Columbus did." 

In the above example, T Reinforced and answered the question, but did not 
Probe . 

e.g., (T adds information) 

T: "Who is the president of the U. S. A.?" 

P: "Mr. Johnson." 

T: "Yes, he is a Texan who was vice-president under Mr. Kennedy." 

In example 2, T Reinforced and added information, but did not Probe . 

COLUMN SIX : T NONRESPONSE OR SAYS^ANSWER_lgjmOITO This is a No-Probe 

Column. 

a) T No ’“Response ; P makes a comment or answers a question, and T 
does not respond to P verbally --he ignores the P response. 

b) T says P*s Answer is Wrong : Once P has responded, T may say: 

"No, you are wrong; incorrect; of course not." When this occurs 
code it as (X) in column 6. Usually T will say (X) then (A) 
answer the question himself (in which case put a (l) in column 

5 for T adds information) or (b) he may Probe by means of one 
of the techniques in columns 7-12. 

I 

COLUMN EIGHT : T CLARIFIES : (P is asked to clarify his &j,nswer): In 

Clarification , T is asking for more information (e.g., "Can you add to 
that?") or for more meaning (e.g., "What do you mean by 
trying to get P to extend the meaning of his response. Clarification 
then includes: (a) WHAT questions 

(b) HOW questions 

(c) WHEN questions 

Note: WHY questions are to be classified as Critical Awareness of Ques- 

tions. (col. 9). In Clarification T asks P to rephrase or clarify his 
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comment , 


answer or question. T may 


(a) 


"What do you mean?" 


(b) 


"Tell us more." 


(c) 


"Please rephrase/clarify." 


(d) 


"Explain further . " 



say: 

(e) "How would it be done?" 

(f) "In what way?" 

(g) "Do you mean...?" 

(h) "Can you give an example 

of that?" 



Three other types of Clarification may occur: 

(1) T asks P to summarize/sum up/tie together/etc. , the discussion up 

to that point, e.g., (a) "Can you sum up the discussion on this 

issue?" 

(b) "What then is the basic point that has 
been made about (this problem)?" 

(2) When P asks a question, T may respond by asking P to break the ques- 
tion down into parts to be considered one at a time, e.g., 

(a) "Can you break your question down into 

parts?" 

(b) "Are you asking one or several questions 

here?" 

T is attempting to get P to Clarify the problem or define the issue so that 
it may then be analyzed or answered. 

(3) When T asks P (following a P question) where one might look for the 

^ answer: e.g., 

"Where would we look for the answer?"; "What sorts of things 
(problems, issues, areas of subject matter) would we have to 

look at first?" 

COLUMN NINE: T CRITICAL AWARENESS : T is trying to get P to become 

critically aware of his assumptions, attitudes, opinions. T is therefore 
asking P to justify rationally his response. T may say: 

a) "Can you support that?" 

b) "Why do you believe that?" 

c) "What are you/we assuming here?" 

d) "Have you oversimplified this - is there more to it?" 

e) T 'quizzical' response: e.g., T may say: Oh? which implicitly 
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asks P to justify his response. In addition T may repeat the 
P response as a question: i.e., 

P: "The butler did it." 

Ti "The butler?" 

f)T may respond to a P response by saying: "Are you sure?" In 
effect, T is asking P to reflect on his response to make sure 
that he is not overlooking or leaving anything out --T is asking 

P to analyze critically his response. 

Note: Both (e) and (f) above may appear to some extent to be class** 

ifiable as one or another of the Probing techniques. To avoid confusion, 
they have arbitrarily been classified as Critica l Awareness examples. 

If we subsequently find other specific T responses that are doubtful, 
^rite them down and submit them to the Research Assistant supervising 
the rating . He will classify them and incorporate them into an appendix. 

Remember, the basic function of a T Critical Awareness Probe to a 
pupil response is to get P to justify his answer. It also includes 
questions that ask P to identify his assumptions. "Why" questions are 
included in this category . 

COLUMN TEN : REDIRECT : Following a P response, T may turn to another 

P and say: 

a) "Do you agree?" 

b) MCan you help him/her out?" 

c) "Can you add to that?" 

d) "What has he/she forgotten?" etc. 

In Redirect, T merely changes the direction of partic ipation by turning 
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to another P. Most frequently, T will use Redirect in conjunction 



with another Probing technique, e.g., 

P^ response: "Sex is funny." 

T response: "Fine, can you add to that. Bill?" 

In the above example, T reinforced P, then used Redirect by Clarify 
(asking for more information and meaning). In this case, score both 
columns. Remember redirect occurs immediately after the P^ response; 
if it does not, then T has asked another question (direct solicit) of 
P^ and has not used redirect. 

COLUMN ELEVEN : T PROMPT : Assume that T is trying to Probe P, but he 

does not fully respond. Rather than accepting this "I don't know" 
type of response, T leads him by asking another question. 

Column Twelve: T REFOCUS : This includes recentering ( a situation where 

P makes an irrelevant response, and T asks P how his response is re- 
lated to the issue at hand). Refocus is signalled by T phrases such as: 

a) "What are the implications of your answer for . ..?" 

b) "How does this relate to/tie into . . . ?" 

c) "If we look at it this way . . . what is your answer then?" 

d) "How would someone (or a specific person or group) answer 

this question?" 

e) "How is your answer related to and different from the position 

of .... on this issue?" 

Refocus will tend to occur when T wants to emphasize a particular 
facet of a problem, or when P has given a high quality answer, and T, 
wishing to Probe , asks P to spell out the implications of the answer 
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for related issues. Refocus as Probe only occurs when T gets P or ^ 

the class to relate , tie in , or give im plications for the previous 
P response. 

Note: The following columns (13, 14,15) relate to T technique 

as opposed to T res ponse styles fls in the above columns. Thus 
they are separated from the rest of the code sheet by a double line. 

COLUMN THIRTEEN ; T ENCOURAGES ALTERNATIVES ; The opposite of Encour- 
aging Alternatives (or divergent thinking) is convergent thinking 
where T leads P to converge on the single, right, correct answer. 

In encouraging alternatives, T gets P or the class to list as many 
possible or probable solutions /reasons/ answers /as possible. Here 

speculation on a broad front is encouraged by T. It is akin to ^ 

brainstorming where you get as many responses as possible, ignoring 

quality of response. If T stops to discuss and analyze each response 

by the class, then he is NOT Encouraging Alternatives . The analysis 

part comes after the list is on the board. Here T goes on to get 

the class to evaluate each of the listed possibilities. 

COLUMN FOURTEEN ; T SUMMARIZES FOR CLASS ; Instead of asking P to 
sum up, T summarizes by saying in effect; "This is what we have 
been doing for the past few minutes." 

COLUMN FIFTEEN ; T USES ROLE PLAY ; T asks P to put himself -herself / 
themselves in someone else's shoes. The intent here is to get P to 

» 

identify with the situation so as to suppress superficial answers) . 

Thus T may say: 

"Pretend you are What would you do/did/say, etc.? ^ 
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RATER'S CODE SHEET 



Page^/ 



SEGMENT RATED FROM: TO: 

TAPE BOX NUMBER 

RATER NAME AND DATE 

RATER EVAL. OF TAPE QUALITY: 

SOUND: VIDEO: 

TYPE OF LESSON L.D. 
AND TIME I.S. 

G.W, 



INTERN NAME: 

INTERN SEX: 

TAPE SESSION NO: 
DATE OF TAPING: 
SUBJECT MATTER: 



TOTAL: 



T Rephrases or 
Clarifies Own 
Response 


Pupil Response 


T Reinforces Pupil 


Q) 

O 

U 

0 

s 


T answers quest, 
or adds information 


No Response or T 
says answer is X 


Probe 


T Clarifies 


T Critical 
Awareness 


4J 

u 

0) 

u 

•H 

& 

H 


T Prompt 


T Refocus 


T encourages 
alternatives 


T summarizes for 
class 


1 T uses Role Play 1 


(1) 


(2) 


(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


(9) 


(10) 


(11] 


(12 


) (13) 


(14) 


:i5) 
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One week prior to training and taping, all models received the 
following directions. They were asked to be ready to teach a five- 
minute discussion lesson incorporating the Probing techniques. 

MODEL'S MANUAL 

BASIC QUESTIONING TECHNIQUES 

Orientation; The basic objective of the upcoming research is to 
help the intern extend the range and quality of his questioning tech- 
niques in such a way that his pupils think more deeply and broadly 

about the problems raised in class. 

We propose to do this by means of models. We would like you to 
teach two or three five-minute lessons in your subject-matter area to 
four pupils . In each lesson your goal will be to demonstrate Probing 

techniques to the best of your ability. Try to inc lude as many in- 

stances of the desired behaviors as possible . The more frequently you 
are able to demonstrate the techniques, the better it will be in terms 
of training the interns. Naturally, there are limits to this. Too 
high a frequency of the desired behaviors may jeopardize the educa- 
tional quality of the lesson. 

The following discussion of specific objectives and the actual 
techniques we would like you to model, are written up as they will be 
presented to the interns. 

At this point, the instructions given on pp.lSSff., Appendix g» 
were attached. 
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INTERN VIDEOTAPE QUESTIONNAIRE 
EXPERIMENT III 
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INTERN VIDEOTAPE QUESTIONNAIRE 



Name of Intern (Print) 

Sex of Intern 

Subject Taught 

Date 

PLEASE BE SURE TO ANSWER EVERY QUESTION . 

My Supervisor today was: Mr. Orme 

Mr. Berliner 

Mark each statement in the right margin according to how much you agree 
or disagree with it. Write +19 +2, +3» or, -1, -2, -3 depending on how 
you feel in each case. 

+1 I agree a little -1 I disagree a little 

+2 I agree on the whole -2 I disagree on the whole 

+3 I agree very much -3 I disagree very much. 

1. In general, viewing the tapes with the supervisor was an enjoyable 
experience for me. 

2. I now have a clear idea of what is meant by probing a pupil 
response. 

3. I feel that using probing techniques in the classroom is educationally 
sound. 

U . The most effective part of the total training experience today 
was : - 



5 . The least effective part of the total training experience today 
was: 
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Below are a number of opposite terms* Rate the supervisor with whom you 
worked today on each pair of terms* Place an (X) in the bracket that 
corresponds to your assessment of the supervisor. 

\ 

\ Examples; Supposing your supervisor was very tall you would score 

1 him as follows: 

short ( )-( )-( )-( )-( )-( )- (X) tall 



Be sure to respond 


to 


» every set 


of 


‘ terms. 










skillful 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


\m skillful 


considerate 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


inconsiderate 


cold 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


warm 


likable 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


imlikable 


irresponsible 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


responsible 


interested 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


disinterested 


authoritarian 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


non-authoritarian 


critical 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


constructive 


rewarding 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


punishing 


accepting 


0 


- 


0 


- 


.0 


- 


0 


- 


0 


- 


0 


- 


0 


demanding 


intimidating 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


encouraging 


realistic 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


imrealistic 


flustered 


0 


» 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


calm 


caring 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


imcaring 


friendly 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


- 


0 


imfriendly 
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STANFORD TEACHER COMPETENCE APPRAISAL GUIDE 
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o 



Model 



STANFORD TEACHER COMPETENCE APPRAISAL GUIDE 

Following each modeling tape, the participating pupils were asked 
to rate the model's performance. The data reported below are the scores 
for the model tapes used in the study. A short form of the Guide appears 
on the next page. Items twelve and thirteen were not scored by the pupils. 



STANFORD APPRAISAL GUIDE SCORES FOR THE MODEL TEACHERS (a) 



4J 

u 

0) 

.o 

0 

CO 



1 s.s 

2 E. 



s 

CO 



M 

M 



3 M. M 



k Sc. 

5 S.S 

6 E. 

7 M. 
7 Sc. 



M 

F 

F 

F 

F 



*d 

0) 

4J 

U 

0) 0) 
0 . 1-1 
(0 0 > 
H (O 



3 

2 

3 

2 

2 

1 

2 

2 



Mean Pupil Responses 



S 



0> 

M 

o 

o 

CO 



58.2 

59.2 
60.0 

61.2 
63.0 
62.8 
57.0 
57.0 



8 9 10 11 

5.5 5.8 5.3 5.0 5.8 5.3 5.3 5.3 5.8 5.3 5.0 

5.8 5.5 5.3 5.3 5.8 5.0 5.5 5.5 5.5 5.0 5.3 

5.0 5.0 5.5 5.3 5.8 5.5 5.8 6.0 5.8 5.8 5.0 

6.0 5.2 5.5 5.5 5.3 6.0 6.0 5.8 6.3 5.8 5.0 

5.3 5.5 5.5 5.8 5.5 5.3. 6.5 6.2 6.0 5.5 6.0 

5.5 5.5 5.8 5.8 5.5 5.3 5.8 6.3 6.0 5.8 5.8 

5.0 5.0 5.0 5.0 5.3 5.3 5.3 5.5 5.3 5.8 5.3 

5.0 5.0 5.0 5.0 5.3 5.3 5.3 5.5 5.3 5.8 5.3 



(a) Four pupils rated each tape. The scores reported for each of the 
eleven items are thus mean scores. The total mean scores were computed 

from the raw scores. 

The norms published for the Guide indicate that the scores may be 
interpreted in the manner described in Table A-1. 



INTERPRETATIVE CLASSIFICATIONS FOR SCORES 
DERIVED FROM THE STANFORD APPRAISAL GUIDE 



Per Cent Scoring This High 


Score 


Interpretative Category 


10? 


T 


Truly Exceptional, 




6 


Outstanding 


15? 


5 


Superior 


15? 


k 


Strong 


15? 


3 


Average 


15? 


2 


Below Average 


30? 


1 


WeaJ^ 


0 


0 


Unable to Observe 


THE STANFORD TEACHER 


COMPETENCE APPRAISAL GUIDE 



The purposes of the lesson are clear. 

The aims are neither too easy nor too 
difficult for the pupils. They are 
appropriate, and are accepted by the pupils. 



1. Clarity of Aims 

2. Appropriateness of 
Aims 



3. Organization of the 
Lesson 



U. Selection of Content 



The individual parts of the lesson are clearly 
related to each other in an appropriate way. 
The total organization facilitates what is to 
be learned. 

The content is appropriate for the aims of the 
lesson, the level of the class, and the 
teaching method. 










rj 
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5 . 



Selection of 
Materials 



6. Beginning the 
Lesson 

7. Clarity of 
Presentation 



8. Pacing of the 
Lesson 



9. Pupil Participation 
And Attention 

10. Ending the Lesson 



11. Teacher-Pupil 
Rapport 

12. Variety of Evalua- 
tive procedures 



13. Use of Evaluation 
to Improve Teaching 
and Learning 



The specific instructional materials and 
human resources used are clearly related to 
the content of the lesson and complement the 
selected method of instruction. 



Pupils tome quickly to attention. They direct 
themselves to the tasks to be accomplished 

The content of the lesson is presented so 
that it is understandable to the pupils. 
Different points of view and specific 
illustrations are used when appropriate. 

The movement from one part of the lesson to 
the next is governed by the pupils achievement. 
The teacher ’’stays with the class” and 
adjusts the tempo accordingly. 

The class is attentive. When appropriate the 
pupils actively participate in the lesson. 

The lesson is ended when the pupils have 
achieved the aims of instruction. There is a 
deliberate attempt to tie together the planned 
and chance events of the lesson and relate 
them to the immediate and long range aims of 
instruction. 

The personal relationships between pupils and 
the teacher are harmonious. 

The teacher devises and uses an adequate variety 
of procedures , both formal and informal , to 
evaluate progress in all of the aims of 
instruction. 

The results of evaluation are carefully 
reviewed by teacher and pupils for the purpose 
of improving teaching and learning. 



Appendix L 



THE STATISTICAL NATURE OF 
THE DEPENDENT VARIABLE 



PROBING 







i 

I 

I 
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All Other Session 1 31.26 38.66 30.88 42.59 54.17 47,00 

Response Session 2 49,75 46.21 40.00 50,61 52.03 46,80 

Categories 

of Probing Session 3 52.17 54.36 40.00 49.80 48.03 45.16 



TABLE L-2 



er|c 



RELIABILITIES OF THE DIFFERENCE SCORES 

USED IN THE ANALYSES OF CONVARIANCE (a) 

(N = 32U) 



Type of Difference Score 


variaoxe 


(Sj - 


(Sg - s^) 


(Sj - Sg) 


Total Pupil Responses 


.99 


.98 


.98 


(1) Pupil Questions 


.63 


.53 


.58 


(2) Pupil Comments 


.8U 


.87 


.85 


(3) Pupil Answers 


.96 


.95 


.92 


Intern Repeats 


.98 


.97 


.95 


Total Non-Probes 


.99 


.98 


.98 


(l) Interns Answers 


.79 


.78 


.68 


(2) Intern No Response 


.88 


.88 


.85 


Total Probes 


.98 


.98 


.97 


(1) Clarification 


.96 


.95 


.91 


(2) Critical Awareness 


.92 


.93 


.89 


(3) Redirection 




.95 




(U) Prompting 


.87 


.89 


.88 


(5) Refocus 


.8U 


.81 


.80 



Ta7 



Reliability of the Difference Score = ^dd ^xx ~ ^xy 



1 - r 



xy 



where, r = inter-rater reliability 

’ XX 



r = session to session correlation for scores 
xy 

on a given variable (McNemar, 19^2, pp. 155-l6l). 
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